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EXECUTIVE SUMMARY 



INTRODUCTION 

This report summarizes the results of the Year 3 evaluation study of the Students and Teachers 
Accessing Real-time Knowledge (STARK) Program in the West Orange Consolidated Independent 
School District (CSID). The overall purpose of the evaluation was twofold: (a) to provide formative 
evaluation data to the participant schools to serve as a basis for improvement planning and as 
documentation of their accomplishments to demonstrate progress; and (b) to provide cumulative 
evidence of the implementation progress and outcomes of the participant schools as well as 
identification of exemplary programs. 

The context for the evaluation consists of 8 schools receiving “Foundation” training developed by 
the Brazos-Sabine Connection and integration training provided by West Orange Collaborative. The 
Foundation training deals directly with developing skills in using technology in the classroom. The 
integration training builds on this and emphasizes using technology in the classroom to support 
student-centered teaching methods that promote higher-level learning outcomes. 

The present 2003-2004 (Year 3) evaluation data were collected from 8 schools over the 
academic year. The data were compared to 2002-2003 (Year 2) and the 2001-2002 (Year 1) baseline 
conditions to gauge program implementation progress. 

RESEARCH QUESTIONS 

The STARK Program evaluation was again structured around five primary research questions 
that focused on classroom practices, degree and type of technology use, academically focused time, 
student engagement, student achievement, and school climate. Also of interest were teacher ability 
with, use of, and attitudes toward technology. 

1 . To what degree and how is technology integrated with classroom instruction by teachers in 
general and those who have received Foundation and/or integration training through the 
Enhanced Learning Academy? 

2. To what degree do integration-trained teachers use methodologies that, consistent with STARK 
Program goals, stress higher-order learning and student-centered learning activities? 

3. To what degree have teachers acquired the technology skills specified in the Texas state 
standards? 

4. What are school outcomes in school climate and teacher uses of and attitudes toward 
technology? To what extent do these variables (a) reflect Foundation training and STARK 
Program goals, and (b) correlate with one another and with implementation success? 

5. What factors appear most instrumental in determining schools’ success at achieving the goals 
and overall implementation of the STARK Program? 

DESIGN 

The evaluation was based on both quantitative and qualitative data collected from classroom 
observations and teacher surveys. Participating schools were the 8 schools receiving “Foundation” 
training developed by the Brazos-Sabine Connection and integration training provided by West 
Orange Collaborative. 
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INSTRUMENTATION 



Six instruments were used to collect the evaluation data: three classroom observation measures 
and three teacher surveys. These instruments have been fully developed and validated. They are as 
follows: 

Classroom Observation Measures 

Observations were made of targeted classes (scheduled visits) using three instruments. 

• School Observation Measure (SOM): Examines frequency of usage of 24 instructional strategies. 

• Survey of Computer Use (SCU): Examines availability and student use of technology and software 

applications. 

• Rubric for Student-Centered Activities (RSCA): Rates the degree of learner engagement in cooperative 

learning, project-based learning, higher-level questioning, experiential/hands-on learning, student 
independent inquiry/research, student discussion, and students as producers of knowledge using 
technology. 

Surveys 

• Teacher Technology Questionnaire (TTQ): Collects teacher perceptions of computers and technology. 

• School Climate Inventory (SCI): Assesses school staff perceptions of school climate on seven 

dimensions: Order, Leadership, Environment, Involvement, Instruction, Expectations, and 
Collaboration. 

• Technology Skills Assessment (TSA): Assesses the perceived technological abilities of the teachers in 

these areas: Computer Basics, Software Basics, Multimedia Basics, Internet Basics, Advanced Skills, 
Using Technology for Learning, and Policy and Ethics. 



PROCEDURE 

Data for this evaluation study were collected in spring 2004. The SOM, SCU and RSCA were 
completed for the targeted observations. These consisted of prearranged one-hour sessions in which 
teachers who participated in the program’s technology training demonstrated a prepared lesson using 
technology. Observation forms were completed for each 1 5 minutes of the lesson. A total of 54 
classroom visits were completed. The teacher surveys (TTQ, SCI, and TSA) were administered in 
spring 2004 during faculty meetings at each school. 

RESULTS 

Below is a brief summary of the results grouped by Classroom Observation Results and Survey 
Results. 

Classroom Observation Results 

The data for 54 classroom observations were collected with SOMs, SCUs, and RSCAs during 
prearranged sessions in which teachers implemented a lesson using technology. Results from each 
measure are described in the section below. Data were also compared based on the number of years 
of teacher participation in the program, public vs. private school, and grade level. 

School Observation Measure (SOM®) 

The greatest significant difference during Year 3 as compared to Year 1 , was increased 
classroom use of “technology as a tool for learning” and as a means of “instructional delivery”. Year 3 
results also revealed significantly greater use of “integration of subject areas” and “higher-level 
questioning” during classroom instruction. Overall data suggest that the STARK program continues 
to have a positive impact on increasing the teacher’s integration of technology into instruction and 
demonstration of higher levels of academically focused class time and student attention. 
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Survey of Computer Use (SCU®) 

The SCU results suggest that progress was made between Year 1 and Year 3 with regard to 
technology integration efforts. Specifically, In Year 3 students more frequently used two production 
tools - draw/graphics and presentation - and more frequently used the Internet for research. In 
addition, the Year 3 data show that teachers were more capable of integrating student use of 
computers into their instruction in meaningful ways. 

Rubric for Student-Centered Activities (RSCA) 

The RSCA results reveal a positive trend in the quality of strategy implementation. This is 
demonstrated by all Year 3 mean scores being above average. . Although the differences were not 
statistically significant, the fact that the mean scores for five of the seven RSCA strategies were higher 
than in previous years indicates continued growth toward teacher competency with non-traditional 
instructional strategies. The most striking RSCA result is the increased usage of technology to support 
student-centered activities. Also noteworthy, is the higher quality implementation of cooperative and 
project-based learning by teachers with three years of program participation as compared to teachers 
with only one year of participation. These results suggest that the STARK Program is successfully 
preparing teachers to implement above average student-centered activities. 



Survey Results 

Three surveys (SCI, TTQ, and TSA) were administered to the teachers during faculty meetings 
held at each school in spring 2004. The responses received were then compared with the responses 
received in the two prior years. The results of these comparisons are summarized below. 

Teacher Technology Questionnaire (TTQ) 

A significantly higher level of teacher agreement was demonstrated for Year 3 as compared to 
Years 1 and/or 2 for all TTQ areas. The greatest percent of Year 1 to Year 3 change was for teacher 
“readiness to integrate technology” (Year 1 = 49.4%; Year 3 = 77.2%). Also noteworthy was an 
increase in agreement with the “impact [of technology] on classroom instruction” items (Year 1 = 
44.0%; Year 3 = 64.3%). These results provide further evidence supporting the positive impacts of 
the STARK Program. 

School Climate Inventory (SCI) 

The SCI responses from 31 2 teachers remained fairly positive across Years 1 , 2, and 3. There 
was slight Year 1 to Year 3 increase in the “instruction” and “collaboration” dimensions and a non- 
significant decrease on three other dimensions. The results for Environment were significantly (p = 
.01) lower than Year 1 results. The mean score, however, was only one percentage point below the 
national SCI norms (M= 3.79). Upon examining results by individual items, the highest levels of 
teacher agreement were for the following: “parents are treated courteously” (89.1%), “the 
administration communicates the belief that all students can learn” (88.8%), “teachers use a variety of 
teaching strategies or models” (88.5%), “the faculty and staff share a sense of commitment to the 
school goals” (86.5%), and “low achieving students are given opportunity for success in this school” 
(86.5%). The item having the lowest level of agreement was “students participate in solving the 
problems of the school” (30.1%). 

Technology Skills Assessment (TSA) 

Year 3 results showed increases in teacher confidence in completing technology-related tasks, 
specifically related to “Internet basics” (Year 1 = 24.5%; Year 3 = 49.8%), “software basics” (Year 1 = 
45.2%; Year 3 = 66.5%), and “using technology for learning” (Year 1=11 .3%; Year 3 = 30.4%). Only 
one area showed a slight Year 1 to Year 3 decrease (2 percentage points) in teacher confidence: 
“multimedia basics” (Year 1 = 48.4%; Year 3 = 46.4%). Inferential analyses revealed significant 
differences between years on all dimensions. Most notable is the Year 1 (M= 1 .52) to Year 3 (M = 
2.07) increase in teacher confidence in using technology for learning, which had an impressive Effect 
Size of +0.90. 



Page 3 of 56 




CONCLUSIONS 

The conclusions of the present study will be presented in association with each of the major 
research questions in the respective sections below. 

To what degree and how is technology integrated with classroom instruction by teachers in 
general and those who have received Foundation and/or integration training through the 
Enhanced Learning Academy? 

The observation data revealed significant Year 1 to Year 3 increases in teacher use of 
computers for instructional delivery and in student use of computers as a learning tool. Specifically, in 
nearly three-quarters (72.2%) of the observations, technology was used as a means of delivering 
instruction through the use of educational software. Also impressive was student use of technology 
as a tool to enhance learning, which was seen in nearly two-thirds (61 .1%) of the visits. Another 
noteworthy increase is seen in RSCA data, which revealed that Year 3 teachers more frequently 
supported student-centered activities with technology. Not only were teachers more frequently 
integrating technology into their instruction, they were able to use a greater variety of software (Year 1 
= 6 of 14 software applications; Year 3 = 11 of 14) and able to do so at a significantly more meaningful 
level. Specifically, during Year 1 “somewhat meaningful” to “meaningful” computer activities were only 
“frequently” to “extensively” observed in 5.6% of the classroom visits as compared to 32.1 % of the 
Year 3 visits. Improvements occurred in the frequency, variety and quality of technology integration 
activities, and significant increases in the types of computer activities were observed with respect to 
three applications: draw/graphics, presentation, and Internet browsers. These results are highly 
suggestive that the STARK Program is effectively impacting technology integration efforts in the target 
schools. Continued professional development is needed, however, to expand teacher ability to 
engage students in use of technology as a learning tool. 

To what degree do integration-trained teachers use methodologies that, consistent with 
STARK Program goals, stress higher-order learning and student-centered learning activities? 

The Year 3 results revealed consistent and often significant increases in the frequency with 
which activities that foster critical thinking and student engagement were observed. Most striking 
were significant increases in the percent of visits in which the teachers utilized “higher-order 
questioning” (Year 1 = 35.2%; Year 3 = 59.3%) and had students use “technology as a learning tool” 
(Year 1 = 18.5%; Year 3 = 61 .1%). Although no significant increases were found in the quality with 
which teachers implement student-centered activities, it is promising to note that all Year 3 RSCA 
mean scores were above average - with five being higher than Year 1 ratings. Of concern, though 
was the limited use of “independent inquiry/research on the part of the students” which was only seen 
in 1 1 .1% of the observations for both Years 1 and 3. The data indicate that teachers were better able 
to create classroom environments that engaged students in higher-order thinking and learning. 
Professional development efforts should, however, focus on better preparing teachers to increase the 
quality and frequency with which they are utilized. 

To what degree have teachers acquired the technology skills specified in the Texas State 
standards? 

Impressive Year 3 TSA results showed that teacher confidence on all seven categories of 
technology skills continued to significantly increase as compared to Years 1 and 2. The greatest 
difference ( ES= +0.90) was seen in teacher confidence in using “technology for learning.” Between 
Year 1 and Year 3, the percentage of teachers who felt they could “very easily” implement this 
strategy increased from 1 1 .1% to nearly one-third (30.4%). The majority of the teachers expressed 
confidence that they could very easily complete basic computer and software tasks and that they 
understood the policy and ethics of technology use, and nearly half felt very confident when using the 
Internet and multimedia software. The percent of teachers who indicated they could very easily 
complete advanced tasks increased to nearly the same degree as those who could use technology for 
learning (Year 1 = 11 .8%; Year 3 = 30.0%). These results are a strong indication of successful 
teacher training through the STARK Program. 
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What are school outcomes in school climate and teacher uses of and attitudes toward 
technology? To what extent do these variables (a) reflect Foundation training and STARK 
Program goals, and (b) correlate with one another and with implementation success? 

Results from the Year 3 School Climate Inventory were mixed, with four mean scores being 
higher than Year 1 and three being lower - only one to a significant degree (Environment - Year 1 M = 
3.94; Year 3 M= 3.78). Even with lower SCI ratings, the environment of the participating schools was 
still positive enough to support significant improvements in school-wide technology integration efforts. 
This was evidenced in the significant increases in frequency of classroom use of technology and 
teacher ability to create and implement meaningful computer activities. Positive teacher attitudes 
were also reflected in the TTQ results, which showed significantly higher teacher readiness to 
integrate technology and more confidence to conduct classes that have students use technology. 
Additionally, there was nearly unanimous teacher agreement that their district/schools provided 
necessary technical and overall support for the technology program. These findings suggest that 
although slight shifts may occur in school climate, the district is supportive of the technology initiative 
and the STARK Program had a positive impact on teacher readiness to integrate technology and 
teacher belief that technology has a positive impact on classroom instruction. 

What factors appear most instrumental in determining schools’ success at achieving the goals 
and overall implementation of the STARK Program? 

Below is a list of key factors from the Year 3 study that appeared to have influenced the progress 
being made toward achievement of the STARK Program implementation and goals. As seen, the 
factors address key elements that are critical for program success: 

Significant differences in instructional practices 

- Increased use of student-centered learning 

- Increased use of computers for instructional delivery 

- Increased use of technology as a learning tool 

Significant differences in the quality of instructional practices 

- Increased use of meaningful computer activities 

Significant differences in teacher attitudes and beliefs regarding technology integration 

- The majority of teachers felt ready to integrate technology 

- The majority of teachers believed technology positively impacts classroom instruction 

- The majority of teachers agreed technology efforts are well-supported 

Significant differences in teacher computer skills 

- The majority of teachers reported confidence with technology tasks 



In general, the Year 3 results reflect impressive progress toward achieving the STARK Program 
goals has been accomplished. Specifically, this year’s data revealed significant changes from Year 1 
in that teachers were more technologically competent, more frequently integrated technology into their 
instruction, and more frequently engaged students in meaningful technology-supported activities. 

Even though significant strides have been made, the scope of technology integration was somewhat 
limited with regard to the variety of software used and the overall degree of computer use. In order to 
enhance student experiences and maximize opportunities for improved learning, program expansion 
is recommended. Suggested areas of expansion include: (1) purchasing more computers to equalize 
distribution of access to all students; (2) implementing a computer maintenance and upgrade program 
to ensure up-to-date computers are available for student use; (3) increasing teacher support through 
the use of onsite Technology Coaches who provide personalized, just-in-time training; and (4) 
continuing and expanding current professional development to build teacher capacity to fully utilize 
technology resources. 
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WEST ORANGE COLLABORATIVE 



STARK PROGRAM 

EVALUATION REPORT 



This report summarizes the results of the 2003-2004 evaluation study for the Students and 
Teachers Accessing Real-time Knowledge (STARK) Program in West Orange Consolidated 
Independent School District (CSID). The overall purpose of the evaluation was twofold: (a) to provide 
formative evaluation data to the participant schools to serve as a basis for improvement planning and 
as documentation of their accomplishments to demonstrate progress; and (b) to provide cumulative 
evidence of the implementation progress and outcomes of the participant schools as well as 
identification of exemplary programs. 

The context for the evaluation consists of 8 schools receiving two levels of educational 
technology training. For the first level, teachers completed the online “Foundation” training modules 
designed by the Brazos-Sabine Connection to develop skills in using technology in the classroom. 

The second level involved teachers completing integration training provided during Saturday training 
sessions and through an online course offered by the University of Texas at Brownsville and designed 
by Karon Tarver of West Orange Collaborative. The integration training built upon the foundation 
skills and emphasized employing technology in the classroom to support student-centered teaching 
methods that promote higher-level learning outcomes. 

The present (Year 3) evaluation concentrated on the progress of the 8 schools in achieving 
STARK Program goals during the 2003-2004 academic year. These data were compared to 2002- 
2003 (Year 2) and 2001-2003 (Year 1) baseline conditions to gauge program implementation 
progress. Suggestions for improvement are offered on the basis of the findings. Specific evaluation 
questions that guided the methodology, data collection, and reporting are listed in the next section. 
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RESEARCH QUESTIONS 



The STARK Program evaluation was structured around five primary research questions that 
focused on classroom practices, degree and type of technology use, academically focused time, 
student engagement, student achievement and school climate. Also of interest were teacher ability 
with, use of, and attitudes toward technology. The detailed questions are listed below: 

• To what degree and how is technology integrated with classroom instruction by teachers in 
general and those who have received Foundation and/or integration training through the 
Enhanced Learning Academy? 

• To what degree do integration-trained teachers use methodologies that, consistent with 
STARK Program goals, stress higher-order learning and student-centered learning activities? 

• To what degree have teachers acquired the technology skills specified in the Texas state 
standards? 

• What are school outcomes in school climate and teacher uses of and attitudes toward 
technology? To what extent do these variables (a) reflect Foundation training and STARK 
Program goals, and (b) correlate with one another and with implementation success? 

• What factors appear most instrumental in determining schools’ success at achieving the 
goals and overall implementation of the STARK Program? 

EVALUATION DESIGN AND MEASURES 

The evaluation was conducted during the spring semester of the 2003-2004 academic year. The 
evaluation design for Year 3 reflected those of the Year 2 study and the Year 1 baseline study to 
enable comparative analyses between the years. The Year 3 design also included analysis of 
observation data by year(s) of teacher participation, source of training, public vs. private school, and 
grade level. Both quantitative and qualitative data were collected at each of the 8 schools by trained 
observers (e.g., retired teachers, district staff). The observers spent the major part of their time visiting 
classrooms (using three instruments to be described below), but also administered surveys to 
teachers. A description of the instruments and data collection procedures is presented below. 

Instrumentation 

Six instruments were used to collect the evaluation data: three classroom observation measures 
and three teacher surveys. These instruments have been fully developed and validated. They are as 
follows: 
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Classroom Observation Measures 

Trained observers conducted classroom visits to collect frequency data regarding observed 
instructional practices. The visits were targeted or scheduled in advance with teachers randomly 
selected from those who participated in the program’s technology training. Selected teachers were 
instructed to deliver a lesson that integrates the use of technology. The data collection instruments 
were the School Observation Measure (SOM), the Survey of Computer Use (SCU), and the Rubric for 
Student-Centered Activities (RSCA ). The SOM collected data regarding overall classroom activities, 
the SCU collected data regarding student use of computers, and the RSCA captured more detailed 
information about student-centered activities during the targeted observations. These classroom 
observation instruments are described below. 

SOM. The SOM was developed to determine the extent to which different common and 

alternative teaching practices are used throughout an entire school or during a targeted lesson (Ross, 
Smith, & Alberg, 1999). During this evaluation, it was used to record observations of classroom 
instruction during prearranged one-hour sessions in which randomly selected teachers demonstrated 
a prepared lesson for which they were asked to use technology. The observers recorded classroom 
events and activities descriptively, not judgmentally. Notes forms were completed every 1 5 minutes 
of the lesson to record the use or nonuse of 24 target strategies and the degree to which a high level 
of academically focused class time and a high level of student attention/interest was observed. At the 
conclusion of the one-hour visit, the observer used a SOM Data Summary Form to summarize the 
frequency with which each of the strategies was observed. The frequency is recorded via a 5-point 
rubric that ranges from (0) Not Observed to (4) Extensively. 

To ensure the reliability of data, observers received a manual providing definition of terms, 
examples and explanations of the target strategies, and a description of procedures for completing the 
instrument. The target strategies include traditional practices ( e.g ., direct instruction and independent 
seatwork) and alternative, predominately student-centered methods associated with educational 
reforms (e.g., cooperative learning, project-based learning, inquiry, discussion, using technology as a 
learning tool). The strategies were identified through surveys and discussions involving policy 
makers, researchers, administrators, and teachers as those most useful in providing indicators of 



Expanded Rubric (ER) in Yr.1 
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schools’ instructional philosophies and implementations of commonly used reform designs (Ross, 

Smith, Alberg, & Lowther, 2001). 

After receiving the manual and instruction in a group session, each observer participated in 
sufficient practice exercises to ensure that his/her data are comparable with those of experienced 
observers. In a reliability study (Lewis, Ross, & Alberg, 1 999), pairs of trained observers selected the 
identical overall response on the five-category rubric on 67% of the items and were within one 
category on 95% of the items. Further results establishing the reliability and validity of SO l/P are 
provided in the Lewis etal. (1999) report. 

SCU . A companion instrument to SOM is the Survey of Computer Use (SCU) (Lowther & Ross, 

2001 ). The SCU was completed as part of the SOM observation sessions during which SCU data 
were also recorded in 15-minute intervals and then summarized on an overall data form. 

The SCU was designed to capture exclusively student access to, ability with, and use of 
computers rather than teacher use of technology. Therefore, in its first section, four primary types of 
data are recorded: (a) computer capacity and currency, (b) configuration, (c) student computer ability 
and (d) student activities while using computers. Computer capacity and currency is defined as the 
age and type of computers available for student use and whether or not Internet access is available. 
Configuration refers to the number of students working at each computer (e.g., alone, in pairs, in small 
groups). Student computer ability is assessed by recording the number of students who are computer 
literate ( i.e ., easily used software features/menus, saved or printed documents) and the number of 
students who easily use the keyboard to enter text or numerical information. 

The next section of the SCU records: the types of student computer activities, and the subject 

areas of those activities. The computer activities are divided into three groups based on broad 

categories of software used: production tools, Internet/research tools, and educational software. 

Within each category, more specific types of software are identified. In the Production Tools category, 

the software types include: word processing, databases, spreadsheets, draw/paint/graphics, 

presentation (e.g., PowerPoint ™), authoring (e.g., KidPix ™), concept mapping (e.g., Inspiration), and 

planning (e.g., MS Project ™). In the Internet/research tools category, the software include: Internet 

browser, CD reference materials, and communications (e.g., email, listservs, chat rooms). In the 

Educational Software category, the software types include: drill/practice/tutorial, problem-solving (e.g., 

Riverdeep ™) and process tools (e.g., Author’s Toolkit ™). With this type of recording system, several 
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activities can be noted during the observation of one student working on a computer. For example, if 
a student gathered data from the Internet, created a graph from the data, then imported the graph into 
a PowerPoint presentation, the observer would record three types of software tools as being 
observed: Internet browser, spreadsheet, and presentation. The computer activities and software 
being used are summarized and recorded using a five-point rubric that ranges from (0) Not Observed 
to (4) Extensively observed. The subject area of each computer activity is categorized as: language 
arts, mathematics, science, social studies, other, or none. 

The final section of the SCU is an “Overall Rubric” designed to assess the degree to which the 
activity reflects “meaningful use” of computers as a tool to enhance learning. The rubric has four 
levels: 1 - Low-level use of computers, 2 - Somewhat meaningful, 3 - Meaningful, and 4 - Very 
meaningful. 

RSCA . The Rubric for Student-Centered Activities was developed by CREP (Lowther, Ross, & 
Plants, 2000) as an extension to SOM and SCU. The RSCA was used by observers to more closely 
evaluate the degree of learner engagement in seven selected areas considered fundamental to the 
goals of increasing student-centered learning activities (cooperative learning, project-based learning, 
higher-level questioning, experiential/hands-on learning, student independent inquiry/research, 
student discussion, and students as producers of knowledge using technology). These strategies 
reflect emphasis on higher-order learning and attainment of deep understanding of content and 
whether or not technology was utilized as a component of the strategy. Such outcomes seem 
consistent with those likely to be engendered by well-designed, real-world linked exercises, projects, 
or problems utilizing technology as a learning tool. Each item includes a two-part rating scale. The 
first is a four-point scale, with 1 indicating a very low level of application, and 5 representing a high 
level of application. The second is a Yes/No option to the question: “Was technology used?” with 
space provided to write a brief description of the technology use. The RSCA was completed during 
the SOM/SCU observation periods. 

Teacher Surveys 

Three surveys were used to collect impressions of the STARK Program: the Teacher 
Technology Questionnaire (TTQ), the School Climate Inventory (SCI), and the Technology Skills 
Assessment (TSA). Each of the participating schools administered the surveys at a faculty meeting 
conducted in May 2004. The surveys are described below. 
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HQ. The Teacher Technology Questionnaire is a three-part instrument used to collect teacher 
perceptions of computers and technology. In the first section, teachers rate their level of agreement 
with 20 statements regarding five technology-related areas: impact on classroom instruction, impact 
on students, teacher readiness to integrate technology, overall support for technology in the school, 
and technical support. Items are rated with a five-point Likert-type scale that ranges from (1 ) Strongly 
Disagree to (5) Strongly Agree. Two primary questions are asked in the second section. The first 
asks teachers to rate their level of computer ability as very good, good, moderate, poor, or no ability. 
The next asks teachers to indicate whether they have a home computer, and if they do, whether they 
use the home computer to access instructional materials on the Internet and/or to prepare classroom 
materials. 

SCI . Researchers at the CREP developed the School Climate Inventory in 1989 (Butler & 

Alberg, 1991). The main purpose of the instrument is to assess impacts of reform initiatives in relation 
to seven dimensions logically and empirically linked with factors associated with effective school 
organizational climates. The inventory contains 49 items, with 7 items comprising each scale. 
Responses are scored through the use of Likert-type ratings ranging from strong disagreement (1) to 
strong agreement (5). Each scale yields scores ranging from 7 to 35, with higher scores being more 
positive. Additional items solicit basic demographic information from respondents. 

Face validity of the school climate items and logical ordering of the items by scales were 
established during the development of the inventory (Butler & Alberg, 1991). Subsequent analysis of 
responses collected through administration of the inventory in a variety of school sites substantiated 
validity of the items. Scale descriptions and current internal reliability coefficients on the seven scales 
of the inventory, obtained using Cronbach’s alpha are as follows: 



Page 11 of 56 




School Climate Inventory Internal Reliability and Scale Descriptions 



Scale 


Internal 

Reliability 


Description 


Order 


a=.8394 


The extent to which the environment is ordered and 
appropriate student behaviors are present 


Leadership 


a=.8345 


The extent to which the administration provides instructional 
leadership 


Environment 


a=.8094 


The extent to which positive learning environments exist 


Involvement 


a=.7582 


The extent to which parents and the community are involved 
in the school 


Instruction 


a=.7453 


The extent to which the instructional program is well 
developed and implemented 


Expectations 


a=.7275 


The extent to which students are expected to learn and be 
responsible 


Collaboration 


a=.7417 


The extent to which the administration, faculty, and students 
cooperate and participate in problem solving 



TSA . The Technology Skills Assessment (TSA) is a 56-item survey that includes 50 three-point 
Likert-type questions, six multiple-choice questions, and one training question. The three-point 
questions are designed to assess the perceived technological abilities of the participants, while the 
multiple choice questions are designed to determine the actual knowledge of the participants on 
several areas of technological literacy. All of the questions are arranged into seven categories, which 
are aligned to the International Society for Technology in Education’s (ISTE’s) National Educational 
Technology Standards (NETS) and the Texas Essential Knowledge and Skills (TEKS) Technology 
Applications for Grades 3-5 (see Appendix A). The categories are: Computer Basics, Software 
Basics, Multimedia Basics, Internet Basics, Advanced Skills, Using Technology for Learning, and 
Policy and Ethics. The final question asks participants to specify the amount of training they have 
received through the Brazos-Sabine Foundation and through the West Orange Collaborative 
integration program. 

DATA COLLECTION 

Table 1 reflects summary information regarding the types of instruments described in the 
previous section. 
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Table 1 

Data Collection Summary 



Classroom 

Observations 


SOM 

SCU 

RSCA 


54 

54 

54 


Spring 2004 • Prearranged one-hour sessions in which 

teachers demonstrated a prepared lesson 
using technology - Notes forms were 
completed every 15 minutes of the lesson. 


Surveys 


SCI 


312 


Spring 2004 • Administered during a faculty meeting 








held during May 2004 at each of the 8 




TTQ 


207 


schools 




TSA 


276 





RESULTS 

Classroom Observation Results 

The data for 54 classroom observations were collected with SOMs, SCUs, and RSCAs during 
prearranged sessions in which teachers were asked to implement a lesson using technology. Results 
from each measure are described in the section below. 

School Observation Measure (SOM 13 ) 

As indicated in the description of SOM, the observation procedure primarily focused on 24 
instructional strategies using a five-point rubric (0 = not observed, 1 = rarely, 2 = occasionally, 3 = 
frequently, and 4 = extensively). In an initial analysis, we computed the percentage of times a strategy 
was not observed (rubric category = 0) vs. observed (categories 1-4 combined). These Year 3 results 
were then compared with Year 2 and the Year 1 baseline data. 

As shown in Table 2, “direct instruction” remained the most frequently observed strategy (81 .5%) 
during the third year; demonstrating a slight decrease (5.6 percentage points) in use as compared to 
Year 1 . The second most frequently observed strategy was “computer for instructional delivery” 
occurring in nearly three-fourths of the observations (72.2%) and demonstrating an impressive 
increase of 42.5 percentage points over Year 1 . The third most frequently observed strategy was 
“teacher-as-a-coach/facilitator” demonstrating an increase of 1 1 .0 percentage points over Year 1 . The 
following three strategies were seen in less than 1 0% of the Year 3 visits: “sustained writing” (9.3%); 
“systematic individual instruction” (7.4%); and “parent/community involvement” (5.6%). 
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Table 2 



School Observation Measure (SOM) Proportion of not observed (0) vs. observed (1-4) strategies 



Year 1 (2001-2002) N = 54 
Year 2 (2002-2003) N = 52 
Year 3 (2003-2004) N = 54 




Direct instruction 


13.0 


19.2 


18.5 


87.1 


78.8 


81.5 


Computer for instructional delivery*** 


68.5 


30.8 


27.8 


29.7 


67.3 


72.2 


Teacher as a coach/facilitator 


40.7 


25.0 


31.5 


57.5 


71.2 


68.5 


Technology as a learning tool*** 


81.5 


32.7 


38.9 


16.8 


65.4 


61.1 


Use of higher-level questioning** 


64.8 


40.4 


40.7 


33.4 


53.8 


59.3 


Higher level instructional feedback 


38.9 


40.4 


48.1 


59.3 


55.8 


51.9 


Experiential, hands on learning 


57.4 


42.3 


55.6 


42.6 


57.7 


44.4 


Independent seatwork 


42.6 


48.1 


57.4 


55.5 


51.9 


42.6 


Integration of subject areas** 


85.2 


53.8 


59.3 


13.0 


44.2 


40.7 


Work centers 


74.1 


65.4 


59.3 


26.0 


34.6 


40.7 


Student discussion 


70.4 


59.6 


61.1 


29.7 


38.5 


38.9 


Ability groups 


72.2 


53.8 


70.4 


27.8 


42.3 


29.6 


Cooperative/collaborative learning 


72.2 


59.6 


74.1 


27.9 


36.5 


25.9 


Project-based learning 


70.4 


50.0 


75.9 


27.8 


46.2 


24.1 


Individual tutoring 


85.2 


61.5 


81.5 


11.3 


32.7 


18.5 


Multi-age grouping 


81.5 


78.8 


87.0 


18.5 


21.2 


13.0 


Performance assessment 


81.5 


84.6 


87.0 


14.9 


13.5 


13.0 


Independent inquiry/research 


88.9 


75.0 


88.9 


11.2 


25.0 


11.1 


Sustained reading 


81.5 


80.8 


88.9 


16.8 


17.3 


11.1 


Team teaching 


83.3 


76.9 


88.9 


16.8 


17.3 


11.1 


Student self-assessment 


90.7 


92.3 


88.9 


7.5 


5.8 


11.1 


Sustained writing/composition 


85.2 


78.8 


90.7 


14.9 


19.2 


9.3 


Systematic individual instruction*** 


100.0 


76.9 


92.6 


0.0 


23.1 


7.4 


Parent/community involvement 


90.7 


88.5 


94.4 


7.5 


7.7 


5.6 



*p<.05, **p<.01, ***p<.001 

Note . Sorted from highest to lowest proportion of 2003-2004 Observed 
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Table 3 presents the full, five-category breakdown of the Year 1 , Year 2, and Year 3 SOM 
results. When examining the strategies that were demonstrated frequently to extensively in excess of 
30% of observations, Year 1 data revealed four activities, Year 2 revealed 10 activities, and Year 3 
data revealed 6 activities. Of the 6 activities for Year 3, two were observed in over 50% of visits: 
direct instruction (61 .1%) and computer for instructional delivery (53.7%). The four remaining 
strategies included technology as a learning tool or resource (44.4%), teacher as facilitator coach 
(40.7%), experiential hands-on learning (31 .5%), and work centers (29.7%), 
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Table 3 



School Observation Measure (SOM) Data Summary 



Year 1 (2001-2002) N = 54 
Year 2 (2002-2003) N = 52 
Year 3 (2003-2004) N = 54 




Instructional Orientation 



Direct instruction (lecture) 


01-02 

02 - 03 

03 - 04 


13.0 

19.2 

18.5 


16.7 

3.8 

5.6 


22.2 

5.8 

14.8 


9.3 

38.5 

27.8 


38.9 

30.8 

33.3 


Team teaching 


01-02 


83.3 


1.9 


1.9 


1.9 


11.1 




02-03 


76.9 


0.0 


1.9 


9.6 


5.8 




03-04 


88.9 


1.9 


0.0 


7.4 


1.9 


Cooperative/collaborative learning 


01-02 


72.2 


1.9 


5.6 


11.1 


9.3 




02-03 


59.6 


1.9 


7.7 


11.5 


15.4 




03-04 


74.1 


5.6 


3.7 


9.3 


7.4 


Individual tutoring (teacher, peer, aide, adult 


01-02 


85.2 


5.6 


1.9 


1.9 


1.9 


volunteer) 


02-03 


61.5 


7.7 


9.6 


11.5 


3.8 




03-04 


81.5 


5.6 


5.6 


3.7 


3.7 


Classroom Organization 


Ability groups 


01-02 


72.2 


0.0 


0.0 


1.9 


25.9 




02-03 


53.8 


0.0 


0.0 


1.9 


40.4 




03-04 


70.4 


3.7 


0.0 


1.9 


22.2 


Multi-age grouping 


01-02 


81.5 


0.0 


0.0 


0.0 


18.5 




02-03 


78.8 


0.0 


0.0 


0.0 


21.2 




03-04 


87.0 


0.0 


0.0 


3.7 


9.3 


Work centers (for individuals or groups) 


01-02 


74.1 


1.9 


0.0 


3.7 


20.4 




02-03 


65.4 


1.9 


3.8 


11.5 


17.3 




03-04 


59.3 


3.7 


5.6 


16.7 


13.0 


Instructional Strategies 


Higher level instructional feedback (written or 


01-02 


38.9 


25.9 


13.0 


13.0 


7.4 


verbal) to enhance student learning 


02-03 


40.4 


7.7 


11.5 


23.1 


13.5 




03-04 


48.1 


11.1 


14.8 


13.0 


13.0 


Integration of subject areas 


01-02 


85.2 


3.7 


1.9 


3.7 


3.7 


(interdisciplinary/thematic units) 


02-03 


53.8 


11.5 


7.7 


13.5 


11.5 




03-04 


59.3 


5.6 


11.1 


7.4 


16.7 


Project-based learning 


01-02 


70.4 


0.0 


3.7 


1.9 


22.2 




02-03 


50.0 


0.0 


5.8 


9.6 


30.8 




03-04 


75.9 


1.9 


5.6 


5.6 


11.1 


Use of higher-level questioning strategies 


01-02 


64.8 


11.1 


9.3 


9.3 


3.7 




02-03 


40.4 


5.8 


9.6 


23.1 


15.4 




03-04 


40.7 


14.8 


16.7 


13.0 


13.0 


Teacher acting as a coach/facilitator 


01-02 


40.7 


9.3 


5.6 


25.9 


16.7 




02-03 


25.0 


5.8 


7.7 


34.6 


23.1 




03-04 


31.5 


7.4 


18.5 


18.5 


22.2 


Parent/community involvement in learning 


01-02 


90.7 


0.0 


0.0 


1.9 


5.6 


activities 


02-03 


88.5 


1.9 


1.9 


0.0 


3.8 




03-04 


94.4 


1.9 


0.0 


1.9 


0.0 
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Table 3 continued 




Student Activities 














Independent seatwork (self-paced 


01-02 


42.6 


11.1 


14.8 


11.1 


18.5 


worksheets, individual assignments) 


02-03 


48.1 


1.9 


11.5 


21.2 


17.3 




03-04 


57.4 


11.1 


13.0 


5.6 


13.0 


Experiential, hands-on learning 


01-02 


57.4 


3.7 


3.7 


9.3 


25.9 




02-03 


42.3 


1.9 


7.7 


23.1 


25.0 




03-04 


55.6 


9.3 


3.7 


14.8 


16.7 


Systematic individual instruction (differential 


01-02 


100.0 


0.0 


0.0 


0.0 


0.0 


assignments geared to individual needs) 


02-03 


76.9 


3.8 


3.8 


7.7 


7.7 




03-04 


92.6 


1.9 


1.9 


3.7 


0.0 


Sustained writing/composition (self-selected 


01-02 


85.2 


1.9 


3.7 


5.6 


3.7 


or teacher-generated topics) 


02-03 


78.8 


0.0 


3.8 


3.8 


11.5 




03-04 


90.7 


0.0 


3.7 


3.7 


1.9 


Sustained reading 


01-02 


81.5 


5.6 


5.6 


5.6 


0.0 




02-03 


80.0 


0.0 


5.8 


3.8 


7.7 




03-04 


88.9 


3.7 


3.7 


1.9 


1.9 


Independent inquiry/research on the part of 


01-02 


88.9 


5.6 


0.0 


1.9 


3.7 


students 


02-03 


75.0 


1.9 


5.8 


9.6 


7.7 




03-04 


88.9 


1.9 


1.9 


3.7 


3.7 


Student discussion 


01-02 


70.4 


5.6 


9.3 


11.1 


3.7 




02-03 


59.6 


5.8 


9.6 


23.1 


0.0 




03-04 


61.1 


9.3 


11.1 


13.0 


5.6 


Technology Use 














Computer for instructional delivery (e.g. CAI, 
drill & practice) 


01-02 


68.5 


3.7 


11.1 


5.6 


9.3 




02-03 


30.8 


1.9 


11.5 


21.2 


32.7 




03-04 


27.8 


7.4 


9.3 


24.1 


29.6 


Technology as a learning tool or resource 


01-02 


81.5 


5.6 


3.7 


1.9 


5.6 


(e.g. Internet research, spreadsheet or 


02-03 


32.7 


3.8 


9.6 


13.5 


38.5 


database creation, multi-media, CD ROM, 
Laser disk) 


03-04 


38.9 


5.6 


11.1 


18.5 


25.9 


Assessment 














Performance assessment strategies 


01-02 


81.5 


0.0 


7.4 


1.9 


5.6 




02-03 


84.6 


0.0 


5.8 


3.8 


3.8 




03-04 


87.0 


3.7 


0.0 


7.4 


1.9 


Student self-assessment (portfolios, 
individual record books) 


01-02 


90.7 


1.9 


1.9 


0.0 


3.7 




02-03 


92.3 


0.0 


3.8 


0.0 


1.9 




03-04 


88.9 


0.0 


0.0 


7.4 


3.7 


Summary Items 








Low 


Moderate 


High 


Academically focused class time 


01-02 






0.0 


29.6 


64.8 




02-03 






3.8 


7.7 


76.9 




03-04 






1.9 


20.4 


77.8 


Level of student attention/ 
Interest/engagement 


01-02 






1.9 


35.2 


59.3 




02-03 






3.8 


15.4 


69.2 




03-04 






3.7 


22.2 


74.1 
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Data from the final section of the SOM shows a steady increase in the frequency with which a 
high level of student attention, interest, and/or engagement was observed from Year 1 (59.3%) to 
Year 3 (74.1%). Similarly, the frequency with which a high level of academically focused time was 
observed increased 13 percentage points from Year 1 (64.8%) to Year 3 (77.8%). 

Inferential results: Year 1 vs. Year 3. To determine whether significant overall changes occurred 
between Years 1 and 3, a series of ANOVA’s were conducted on the multiple items of the SOM. As 
shown in Table 4 and Figure 1 there was a significantly higher frequency of occurrence of 4 strategies 
during the Year 3 classroom visits. The Effect Sizes for the four items ranged from +0.47 to +0.95. 
The two items revealing the greatest increases between Year 1 and the Year 3 were “technology as a 
learning tool” (Year 1 M= 0.47; Year 3 M= 1 .87) and “computer for instructional delivery” (Year 1 M = 
0.81 ; Year 3 M = 2.21 ) - both key goals of the STARK program. 

Table 4 

A Summary of SOM Items Showing Significant Differences between Year 1 and Year 3 




Technology as learning tool or resource 


0.42 


1.06 


1.87 


1.69 


20.33 


.000 


+0.95 


Computer for instructional delivery 


0.81 


1.37 


2.21 


1.63 


14.21 


.000 


+0.89 


Integration of subject areas 


0.34 


0.98 


1.17 


1.58 


6.23 


.002 


+0.60 


Use of higher-level questioning strategies 


0.74 


1.20 


1.41 


1.47 


5.69 


.004 


+0.47 



Scale: 0 = Not Observed; 4 = Extensively Observed 
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SOM: Year 1 vs. Year 3 Significant Differences 





□ Year 1 
■ Year 3 



0.42 

1.87 



0.81 

2.21 



0.34 

1.17 



0.74 

1.41 



Figure 1 

Inferential results: Year 3 categorical comparisons. Inferential analyses, Chi-square and 
MANOVA, were also conducted comparing type of school (public vs. private), grade level, and 
number of years of teacher participation in the program. Chi-square analysis was used to identify 
significant differences in the proportion of observed vs. not observed strategies, and MANOVA was 
used to identify any significant differences among the instructional strategies. 

Type of school. No significant differences were revealed regarding type of school (public vs. 
private) for Chi-square or MANOVA analyses. 

Grade Level. Table 5 presents the SOM data broken down separately for pre-K, elementary, 
middle and high school grade level groups. Chi-square analysis revealed significant differences 
between grade levels for 37.5% (9 Of 24) of the instructional strategies. MANOVA analysis also 
revealed significant differences between grade levels F( 75, 65) = 2.02, p < .01 . Follow-up univariate 
analysis yielded significance on 3 of the 24 strategies. The first was “multi-age grouping”, which was 
observed significantly more (p= .001) in high school ( M= 2.0) visits, than pre-middle school ( M= 0.0), 
elementary ( M = 0.1 5), and pre-kindergarten ( M = 0.0) visits. This finding is not surprising, as most 
high school classes are comprised of multi-age students. “Systematic individualized instruction” was 
observed at a significantly higher frequency (p < .05) in pre-kindergarten (M = .88) classes as 
compared to elementary (M= 0.11), middle ( M= 0.0), or high school ( M= 0.0). The third area was 
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“student discussion” with a significantly higher frequency (p < .05) observed in elementary (M = 0.54) 
than in pre-kindergarten. 

Table 5 



Grade Level - Year 3 SOM Proportion of not observed (0) vs. observed (1-4) strategies 

Pre-Kindergarten N = 07 

Elementary School N = 26 

Middle School N = 10 

High School N = 11 



Direct instruction* 


14.3 


11.5 


50.0 


9.1 


85.7 


88.5 


50.0 


90.9 


Teacher as a 
coach/facilitator 


28.6 


20.0 


60.0 


36.4 


71.4 


80.0 


40.0 


63.6 


Computer for 
instructional delivery* 


0.0 


19.2 


66.7 


36.4 


100.0 


80.8 


33.3 


63.6 


Student discussion* 


85.7 


73.1 


40.0 


36.4 


14.3 


26.9 


60.0 


63.6 


Higher level instructional 
feedback 


57.1 


53.8 


30.0 


45.5 


42.9 


46.2 


70.0 


54.5 


Multi-age grouping*** 


100.0 


96.2 


100.0 


45.5 


0.0 


3.8 


0.0 


54.5 


Use of higher-level 
questioning 


57.1 


36.0 


40.0 


45.5 


42.9 


64.0 


60.0 


54.5 


Technology as a learning 
tool 


42.9 


30.8 


40.0 


54.5 


57.1 


69.2 


60.0 


45.5 


Cooperative/collaborative 

learning 


85.7 


80.8 


70.0 


54.5 


14.3 


19.2 


30.0 


45.5 


Ability groups 


85.7 


76.0 


70.0 


54.5 


14.3 


24.0 


30.0 


45.5 


Experiential, hands on 
learning 


14.3 


61.5 


60.0 


63.6 


85.7 


38.5 


40.0 


36.4 


Individual tutoring 


71.4 


92.3 


80.0 


63.6 


28.6 


7.7 


20.0 


36.4 


Independent seatwork** 


100.0 


34.6 


70.0 


72.7 


0.0 


65.4 


30.0 


27.3 


Integration of subject 
areas 


42.9 


65.4 


40.0 


72.7 


57.1 


34.6 


60.0 


27.3 


Project-based learning 


85.7 


80.8 


60.0 


72.7 


14.3 


19.2 


40.0 


27.3 


Work centers* 


14.3 


60.0 


80.0 


72.7 


85.7 


40.0 


20.0 


27.3 


Independent 

inquiry/research 


85.7 


96.2 


90.0 


72.7 


14.3 


3.8 


10.0 


27.3 


Student self-assessment 


100.0 


96.2 


80.0 


72.7 


0.0 


3.8 


20.0 


27.3 


Performance 

assessment 


85.7 


88.5 


90.0 


81.8 


14.3 


11.5 


10.0 


18.2 


Sustained reading 


100.0 


84.6 


80.0 


100.0 


0.0 


15.4 


20.0 


0.0 


Team teaching* 


57.1 


92.3 


90.0 


100.0 


42.9 


7.7 


10.0 


0.0 


Sustained 

writing/composition 


100.0 


88.5 


80.0 


100.0 


0.0 


11.5 


20.0 


0.0 


Parent/community 

involvement** 


71.4 


100.0 


100.0 


100.0 


28.6 


0.0 


0.0 


0.0 


Systematic individual 
instruction** 


57.1 


96.2 


100.0 


100.0 


42.9 


3.8 


0.0 


0.0 



*p<.05, **p<.01, ***p<.001 

Note . Sorted from highest to lowest proportion of Observed for high school 
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Years of Program Participation . Chi-square analysis revealed a significant difference depending 
on number of year of teacher program participation for 21% (5 of 24) strategies. As shown in Table 6, 
program participation had the greatest effect on the frequency of “teacher as coach/facilitator” (1 Year 
= 60.0%; 3 Years = 95.2%) and use of “work centers” (1 Year = 26.7%; 3 Years = 66.7%) 
observations. The other areas of significance included “independent seatwork”, “performance 
assessment”, and “student self-assessment”. Results of MANOVA analysis revealed no significant 
differences between instructional strategies on the basis of years of program participation. 

Table 6 



Years of Program Participation - Year 3 SOM Proportion of not observed (0) vs. observed (1-4) 
strategies 

1 Year N = 15 

2 Years N = 17 

3 Years N = 22 



Teacher as a coach/facilitator** 


40.0 


58.8 


4.8 


60.0 


41.2 


95.2 


Computer for instructional delivery 


46.7 


17.6 


23.8 


53.3 


82.4 


76.2 


Direct instruction 


20.0 


5.9 


6.0 


80.0 


94.1 


72.7 


Work centers** 


73.3 


82.4 


33.3 


26.7 


17.6 


66.7 


Use of higher-level questioning 


66.7 


29.4 


33.3 


33.3 


70.6 


66.7 


Technology as a learning tool 


26.7 


52.9 


36.4 


73.3 


47.1 


63.6 


Experiential, hands on learning 


73.3 


64.7 


36.4 


26.7 


35.3 


63.6 


Higher level instructional feedback 


66.7 


41.2 


40.9 


33.3 


58.8 


59.1 


Ability groups 


86.7 


76.5 


57.1 


13.3 


23.51 


42.9 


Independent seatwork* 


33.3 


76.5 


59.1 


66.7 


23.5 


40.9 


Integration of subject areas 


53.3 


64.7 


59.1 


46.7 


35.3 


40.9 


Student discussion 


60.0 


64.7 


59.1 


40.0 


35.3 


40.9 


Cooperative/collaborative learning 


73.3 


88.2 


63.6 


26.7 


11.8 


36.4 


Project-based learning 


80.0 


82.4 


68.2 


20.0 


17.6 


31.8 


Performance assessment* 


100.0 


94.1 


72.7 


0.0 


5.9 


27.3 


Individual tutoring 


86.7 


88.2 


72.7 


13.3 


11.8 


27.3 


Student self-assessment** 


100.0 


100.0 


72.7 


0.0 


0.0 


27.3 


Sustained reading 


93.3 


94.1 


81.8 


6.7 


5.9 


18.2 


Independent inquiry/research 


93.3 


88.2 


86.4 


6.7 


11.8 


13.6 


Team teaching 


100.0 


82.4 


86.4 


0.0 


17.6 


13.6 


Sustained writing/composition 


93.3 


94.1 


86.4 


6.7 


5.9 


13.6 


Multi-age grouping 


26.7 


82.4 


90.9 


13.3 


17.6 


9.1 


Parent/community involvement 


100.0 


100.0 


90.9 


0.0 


0.0 


9.1 


Systematic individual instruction 


93.3 


88.2 


95.5 


6.7 


11.8 


4.5 



*p<.05, **p<.01 
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Note . Sorted from highest to lowest proportion of Observed for 3 Years participation 



Summary. The greatest significant differences between Year 3 and Year 1 were increased 
classroom use of technology “as a tool for learning" and as a means of “instructional delivery”. Year 3 
results also revealed significantly greater use of “integration of subject areas” and “higher-level 
questioning” during classroom instruction. Overall data suggest that the STARK program continues 
to have a positive impact on increasing the teacher’s integration of technology into instruction and 
demonstration of higher levels of academically focused class time and student attention. 

Rubric for Student-Centered Activities (RSCA) 

As with the SOM and the SCU, data from the RSCA were collected during prescheduled lessons 
in which teachers were asked to use technology. Results address the quality/depth of observed 
strategy applications and the percentage of sessions in which technology was used with the observed 
strategy. Because the RSCA was used in targeted observations of lessons that were to include the 
use of technology to support learning, computer use was expected to be viewed at sometime during 
the lesson. 

Ratings. The analyses examined the rubric rating (1 to 4) for each of the strategies, when the 
given strategy was observed. That is, if the strategy was not seen, the associated rating of “0” was 
excluded from the analysis because it would seriously negatively bias the overall computation of 
quality/effectiveness. The mean rubric ratings for each of the strategies for Years 1 , 2, and 3 are 
presented in Table 7. As shown, the mean ratings for Year 3 ranged from 2.45 to 3.13, which 
continue to indicate moderate to moderately high levels of quality/effectiveness. “Independent 
inquiry/research” (M= 3.13) and “experiential, hands-on learning” (M = 3.12) were assessed as the 
most meaningful strategies followed by “project-based learning” ( M= 3.00). The greatest Year 1 to 
Year 3 increase in ratings (60 percentage points) was demonstrated for “students as producers of 
knowledge” (Year 1 M= 1 .85; Year 3 M= 2.45). Use of only one strategy, “experiential, hands-on 
learning,” showed a slight decrease (17 percentage points) from Year 1 to Year 3 (Year 1 M= 3.29; 

Year 3 M= 3.12). The mean scores for both years, however, represent above average 
implementation of this strategy. 

Inferential results: Year 1 vs. Year 3. Inferential analyses (ANOVA test for independent samples) 
were conducted to compare rubric ratings. No significant differences were revealed. 
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Table 7 



Rubric for Student Centered Activities (RSCA) Item Ratings by Percentage Observed and 
Mean Scores 

Year 1 (2001-2002) N = 52 
Year 2 (2002-2003) N = 54 
Year 3 (2003-2004) N = 54 




Cooperative Learning 


01-02 


29.6 


1.9 


11.1 


9.3 


7.4 


2.75 


3.7 




02-03 


47.1 


15.1 


11.3 


7.5 


13.2 


2.40 


39.6 




03-04 


27.8 


5.6 


3.7 


9.3 


9.3 


2.80 


18.5 


Project-Based Learning 


01-02 


27.8 


3.7 


5.6 


5.6 


13.0 


3.00 


9.3 




02-03 


47.2 


1.9 


13.2 


13.2 


18.9 


3.04 


45.3 




03-04 


25.9 


1.9 


9.3 


1.9 


13.0 


3.00 


24.1 


Higher-Level Questioning 


01-02 


53.7 


18.5 


14.8 


9.3 


7.4 


2.11 


1.9 


Strategies 


02-03 


56.6 


11.3 


17.0 


13.2 


15.1 


2.57 


39.6 




03-04 


61.1 


9.3 


20.4 


16.7 


14.8 


2.61 


38.9 


Experiential, Hands-On 


01-02 


38.9 


1.9 


9.3 


3.7 


24.1 


3.29 


3.7 


Learning 


02-03 


58.5 


3.8 


13.2 


17.0 


24.5 


3.06 


45.3 




03-04 


48.1 


3.7 


7.4 


16.7 


20.4 


3.12 


31.5 


Independent Inquiry / Research 


01-02 


16.7 


7.4 


0.0 


1.9 


7.4 


2.56 


7.4 




02-03 


35.9 


13.2 


3.8 


3.8 


15.1 


2.58 


28.3 




03-04 


14.8 


0.0 


5.6 


1.9 


7.4 


3.13 


13.0 


Student Discussion 


01-02 


33.3 


18.5 


0.0 


5.6 


9.3 


2.17 


1.9 




02-03 


45.2 


9.4 


7.5 


20.8 


7.5 


2.58 


28.3 




03-04 


42.6 


9.3 


9.3 


11.1 


13.0 


2.65 


20.4 


Students as Producers of 


01-02 


24.1 


16.7 


0.0 


1.9 


5.6 


1.85 




Knowledge 


02-03 


43.4 


7.5 


15.1 


5.7 


15.1 


2.65 






03-04 


37.0 


14.8 


3.7 


5.6 


13.0 


2.45 





* Rating scale: 1 = limited application; 4 = Strong application. 



Technology use. As can be seen in Figure 2, technology was used to support RSCA strategies 
to a greater extent in Year 3 as compared to Year 1 . In over 30% of the observations, technology 
was used to support “experiential hands-on learning activities” (31 .5%) and the use of “higher level 
questioning strategies” (38.9%). Teachers also used technology to support “project-based learning” 
(24.1%) and “student discussion” (20.4%) during approximately one-fifth of the observations. 
Technology was used less frequently in conjunction with independent inquiry, as these uses were only 
observed in 1 3% of the visits. 
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Year 1 vs. Year 3: Percent of RSCA Activities* that Used Technology 



40.0 



35.0 



30.0 



25.0 




20.0 



15.0 



10.0 



u.u 


Cooperative 

Learning 


Project-based 

Learning 


Higher-Level 

Questions 


Hands-on 

Learning 


Independent 

Inquiry 


Student 

Discussion 


□ Year 1 


3.7 


9.3 


1.9 


3.7 


7.4 


1.9 


■ Year 3 


18.5 


24.1 


38.9 


31.5 


13.0 


20.4 



‘Students as Producers of Knowledge” is not included because use of technology is a required component. 

Figure 2 



Inferential results: Year 3 categorical comparisons. Inferential analyses were conducted to 
compare rubric ratings according to type of school (public vs. private), grade level (pre-kindergarten, 
elementary, middle, and high), and number of years of teacher participation in the program (1 , 2 or 3 
years). Results are discussed below. 

Type of school . A significant difference was noted only for the “student discussion” dimension. 
The mean rating for observations conducted in public schools (M = 3.05) was significantly higher than 
for observations conducted in private schools (M= 1 .20). 

Grade Level . ANOVA’s were conducted to examine differences due to grade level. No 
significant differences were revealed. 

Years of Program Participation . As shown in Table 8, significant differences were revealed for 
“cooperative learning”, “project-based learning”, and “independent inquiry/research”. Ratings for 
“cooperative learning” and “project-based learning” were significantly higher for teachers having 
participated in the program for 3 years than for teachers participating 2 years. However, ratings for 
“independent inquiry/research” were significantly lower for teachers participating in the program for 2 
years than for all other teachers. 
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Table 8 



Years of Program Participation - Year 3 RSCA Item Ratings by Percentage Observed and Mean 
Scores 



1 Year N = 15 

2 Years N = 17 

3 Years N = 22 




Cooperative Learning 


1 


33.4 


6.7 


6.7 


20.0 


0.0 


2.40 




2 


23.5 


11.8 


5.9 


0.0 


5.9 


2.00 




3 


27.3 


0.0 


0.0 


9.1 


18.2 


3.67 


Project-Based Learning 


1 


20.0 


0.0 


6.7 


0.0 


13.3 


3.33 




2 


23.5 


5.9 


17.6 


0.0 


0.0 


1.75 




3 


31.8 


0.0 


4.5 


4.5 


22.7 


3.57 


Higher-Level Questioning Strategies 


1 


46.7 


13.3 


13.3 


6.7 


13.3 


2.43 




2 


70.6 


11.8 


23.5 


29.4 


5.9 


2.42 




3 


63.6 


4.5 


22.7 


13.6 


22.7 


2.86 



Experiential, Hands-On Learning 


1 


33.3 


6.7 


6.7 


6.7 


13.3 


2.80 




2 


41.2 


5.9 


0.0 


23.5 


11.8 


3.00 




3 


63.6 


0.0 


13.6 


18.2 


31.8 


3.29 



Independent Inquiry / Research 


1 


6.7 


0.0 


0.0 


0.0 


6.7 


4.00 




2 


17.6 


0.0 


17.6 


0.0 


0.0 


2.00 




3 


18.2 


0.0 


0.0 


4.5 


13.6 


3.75 


Student Discussion 


1 


46.7 


20.0 


6.7 


6.7 


13.3 


2.29 




2 


35.3 


5.9 


11.8 


11.8 


5.9 


2.50 




3 


45.5 


4.5 


9.1 


13.6 


18.2 


3.00 


Students as Producers of Knowledge 


1 


40.0 


26.7 


6.7 


0.0 


6.7 


1.67 




2 


23.5 


11.8 


0.0 


5.9 


5.9 


2.25 




3 


45.5 


9.1 


4.5 


9.1 


22.7 


3.00 



* Rating scale: 1 = limited application; 4 = Strong application. 



Summary. The RSCA results revealed a positive trend in the quality of strategy implementation; 
which is demonstrated by all Year 3 mean scores being above average. Although not significant, the 
mean scores for five of the seven RSCA strategies were higher than previous years thus indicating 
continued growth toward teacher competency with non-traditional instructional strategies. The most 
striking RSCA result is the increased usage of technology to support student-centered activities. Also 
noteworthy, is the higher quality with which teachers implemented cooperative and project-based 
learning after three years as compared to one year of program participation. These results suggest 
that the STARK Program is successfully preparing teachers to implement above average student- 
centered activities. 
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Survey of Computer Use (SCU) 

As with the SOM and RSCA, data from the SCU were collected during prescheduled lessons in 
which teachers were asked to use technology. A summary of the observation results for all years is 
provided in Table 9. Data for the 2001 -2002 Computer Configuration, Testing Software, and 
Computers/Digital Tools Used by Students are not listed in the table as the reporting format changed 
from Year 1 to Year 2. 

Computer Configuration. Year 3 SCU data showed similar results to Year 2 in that the majority of 
the classrooms (80%) had two or more computers that were typically considered to be well up-to-date 
(71 .7%) and connected to the Internet (84.9%). Desktop and laptop computers were again the most 
prevalent type of technology used in the classroom as they were seen in approximately 50% of the 
visits (50.9%, 47.2% respectively). The percent of visits in which students displayed a high level of 
computer literacy skills remained consistent (Year 2 = 19.6%; Year 3 = 20.8%), while there was a 
decrease in students demonstrating “very good” keyboarding skills (Year 2 = 25.5%; Year 3 = 9.4%). 

Student Computer Activities by Software used. The Year 3 as compared to Year 1 SCU results 
show a greater variety of software applications being used by the students. During Year 1 , only 6 of 
the 14 (without Testing software) listed software tools were seen in use, whereas 1 1 of 14 were 
observed to some degree during the Year 3 classroom visits. Brief descriptions of the different 
software uses, related subject area content of the activities, and noted significant differences derived 
from Analysis of Variance are presented below. 

Production tools used by students. The Year 3 data show students used each type of production 
tool. Three applications, however, were more frequently used than the others (Table 9). Word 
processing was the most frequently observed software, as students were seen using this tool in 
almost 20% (17%) of the class visits, draw/graphic was seen in 1 5.1%, while presentation software 
was seen in 1 1 .3% of the visits. With regard to subject area, production tools were typically used for 
language arts (22.6%), and mathematics and science (15.1%) activities. The Year 1 to Year 3 
comparative results are presented in Table 10. Of the production tools, students used draw/graphic 
tools (p = 0.02) and presentation software (p < .01 ) significantly more during Year 3 as compared to 
Year 1 . However, as noted earlier, the extent of use was limited in that draw/graphic software was 
frequently or extensively seen in only 1 3.2% of the classroom observations and presentation software 
was only seen to that degree in 5.7%. 
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Internet/research tools used by students. There was a significant Year 1 to Year 3 increase in 
the percent of observations in which students were using the Internet (p < .01). Yet, frequent to 
extensive use of the Internet was only seen in only 7.6% of the visits. This result is not surprising 
considering the district’s limited access to online resources due to available bandwidth. 

Educational software use by students. There was a moderate increase in the use of educational 
software from Year 1 (16.7%) to Year 3 (26.4%); however, the change was non-significant. The 
majority of the educational software activities were focused on language arts (1 7.0%). 

Testing software use by students. Since testing software was not available for Year 1 , and not 
observed in Year 2 , a Year 3 comparative analysis was not conducted. 

Overall Meaningful Use of Computers. The culminating assessment on the SCU was the 
observer’s evaluation of the meaningfulness of the way in which technology was integrated with 
teaching and learning. To do this, they were asked to indicate how often they observed computer 
activities at each level of the rubric; e.g., how often was very meaningful use of computers observed. 
As can be seen in Table 1 1 and Figure 3, significant differences between Year 1 and Year 3 
observations were found on two levels of the rating scale. The direction of the data shows 
progression toward more meaningful applications during Year 3. Specifically, “meaningful use” was 
frequently to extensively observed in 1 7.0% of the Year 3 visits but in only 1 .9% of the Year 1 visits (p 
= .001 ). Likewise, “somewhat meaningful” usage was observed frequently to extensively in 15.1% of 
the Year 3 visits as compared to only 3.7% in Year 1 (p = .03,). 

Inferential results: Year 3 categorical comparisons. Inferential analyses were again conducted 
to compare SCU rating according to the type school (public vs. private), grade level (pre-kindergarten, 
elementary, middle, and high) and the number of years of teacher participation in the program (1 . 2, or 
3 years). Chi-square analysis indicated a significant difference (p = 01 ) in the ratings of student 
keyboarding skills according to the number of years of teacher program participation. 



Page 27 of 56 




Table 9 



Survey of Computer Use (SCU) Data Summary 



Year 1 (2001-2002) N = 54 
Year 2 (2002-2003) N = 51 
Year 3 (2003-2004) N = 53 




Classrooms most frequently had the following number of computers or digital tools. 



None 




2.0 


0.0 


One 




17.6 


20.8 


2-4 




51.0 


45.3 


5-10 




5.9 


30.2 


1 1 or more 




23.5 


3.8 


Classroom computers were most frequently 








Up-to-date 




64.7 


71.7 


Aging, but adequate 




29.4 


26.4 


Outdated/limited capacity 




2.0 


1.9 


No computers were observed 




2.0 


0.0 


Classroom computers were most frequently 








Connected to the Internet 




84.3 


84.9 


Not connected to the Internet 




13.7 


13.2 


No computers were observed 




2.0 


0.0 


Student Computer Use 








Classroom Computers or digital tools were most frequently used by: 






Few (less than 1 0%) students 




27.5 


22.6 


Some (about 1 0-50%) students 




13.7 


20.8 


Most (about 51 -90%) students 




2.0 


9.4 


Nearly all (91 -1 00%) students 




35.3 


26.4 


Students did not use computers 




17.6 


20.8 


Students most frequently worked with computers or digital tools: 






Alone 




45.1 


49.1 


In pairs 




7.8 


15.1 


In small groups 




15.7 


7.5 


Students did not use computers 
Student computer literacy skills were most frequently: 




27.5 


24.5 


Poor 




7.8 


7.5 


Moderate 




27.5 


20.8 


Very good 




19.6 


20.8 


Not observed 




41.2 


45.3 


Student keyboarding skills were most frequently: 








Poor 




9.8 


9.4 


Moderate 




5.9 


11.3 


Very good 




25.5 


9.4 


Not observed 




54.9 


64.2 



Note . Results marked as are not listed as they were reported in a different format or not collected in 2001-2002. 
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Table 9 continued 




Desktop Computers 
Laptop Computers 
Personal Data Assistants (PDA) 

Graphing Calculator 

Information Processor (e.g., Alphaboard) 

Digital Accessories [e.g., camera, scanner, 
probes) 



2002-2003 


47.1 


2.0 


5.9 


11.8 


25.5 


2003-2004 


49.1 


9.4 


9.4 


17.0 


13.2 


2002-2003 


52.9 


7.8 


9.8 


9.8 


13.7 


2003-2004 


52.8 


5.7 


7.5 


18.9 


11.3 


2002-2003 


88.2 


0.0 


0.0 


0.0 


2.0 


2003-2004 


96.2 


0.0 


0.0 


0.0 


0.0 


2002-2003 


88.2 


0.0 


0.0 


0.0 


2.0 


2003-2004 


92.5 


0.0 


0.0 


0.0 


3.8 


2002-2003 


90.2 


0.0 


0.0 


0.0 


0.0 


2003-2004 


94.3 


0.0 


0.0 


1.9 


1.9 


2002-2003 


78.4 


0.0 


5.9 


0.0 


5.9 


2003-2004 


88.7 


3.8 


0.0 


1.9 


1.9 



*Note: Item percentages may not total 1 00% because of missing input from some respondents. 




Production Tools 



Word Processing 


2001-2002 


88.9 


0.0 


5.6 


1.9 


3.7 




2002-2003 


76.5 


3.9 


3.9 


0.0 


11.8 




2003-2004 


83.0 


1.9 


1.9 


7.5 


3.8 


Database 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


100.0 


0.0 


0.0 


0.0 


0.0 




2003-2004 


96.2 


0.0 


0.0 


0.0 


1.9 


Spreadsheet 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


90.2 


0.0 


0.0 


2.0 


3.9 




2003-2004 


90.6 


0.0 


1.9 


1.9 


3.8 


Draw/Paint/Graphics 


2001-2002 


98.1 


1.9 


0.0 


0.0 


0.0 




2002-2003 


80.4 


3.9 


2.0 


2.0 


5.9 




2003-2004 


83.0 


1.9 


0.0 


3.8 


9.4 


Presentation (e.g., MS PowerPoint)** 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


74.5 


2.0 


0.0 


5.9 


9.8 




2003-2004 


88.7 


1.9 


1.9 


1.9 


3.8 


Authoring (e.g., HyperStudio) 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


100.0 


0.0 


0.0 


0.0 


0.0 




2003-2004 


94.3 


0.0 


0.0 


1.9 


0.0 


Concept Mapping (e.g., Inspiration) 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


90.2 


0.0 


0.0 


0.0 


3.9 




2003-2004 


94.3 


0.0 


0.0 


1.9 


0.0 


Planning (e.g., MS Project) 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


90.2 


0.0 


0.0 


2.0 


0.0 




2003-2004 


96.2 


0.0 


0.0 


0.0 


0.0 
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Table 9 continued 




Internet/Research Tools 



Internet Browser ( e.g Netscape)** 


2001-2002 

2002 - 2003 

2003 - 2004 


92.6 

74.5 

84.9 


7.4 

0.0 

5.7 


0.0 

2.0 

0.0 


0.0 

3.9 

3.8 


0.0 

11.8 

3.8 


CD Reference (encyclopedias, etc.) 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


86.3 


0.0 


0.0 


3.9 


0.0 




2003-2004 


96.2 


1.9 


0.0 


0.0 


0.0 


Communications 


2001-2002 


100.0 


0.0 


0.0 


0.0 


0.0 




2002-2003 


100.0 


0.0 


0.0 


0.0 


0.0 




2003-2004 


96.2 


0.0 


0.0 


0.0 


0.0 


Educational Software 


Drill/Practice/T utorial 


2001-2002 


83.3 


3.7 


1.9 


0.0 


11.1 




2002-2003 


78.4 


0.0 


2.0 


5.9 


9.8 




2003-2004 


73.6 


5.7 


5.7 


3.8 


5.7 


Problem Solving [ e . g ., SimCity) 


2001-2002 


98.1 


0.0 


0.0 


0.0 


0.0 




2002-2003 


90.2 


0.0 


0.0 


0.0 


2.0 




2003-2004 


94.3 


0.0 


0.0 


1.9 


0.0 


Process Tools (Geometer's Sketchpad, 


2001-2002 


94.4 


0.0 


0.0 


0.0 


3.7 


etc.) 




2002-2003 


90.2 


0.0 


0.0 


2.0 


2.0 




2003-2004 


92.5 


0.0 


0.0 


1.9 


0.0 


Testing Software 


Individualized/Tracked ( e . g ., Accelerated 


2001-2002 












Reader ) 




2002-2003 


88.2 


0.0 


0.0 


0.0 


0.0 




2003-2004 


83.0 


0.0 


3.8 


0.0 


1.9 


Generic 


2001-2002 














2002-2003 


86.3 


0.0 


0.0 


0.0 


0.0 




2003-2004 


86.8 


0.0 


0.0 


0.0 


0.0 



** p <.01 




Production Tools 


2001-2002 

2002 - 2003 

2003 - 2004 


90.7 

27.5 

45.3 


5.6 
11.8 

5.7 


1.9 

37.3 

22.6 


0.0 

11.8 

15.1 


0.0 

7.8 

15.1 


I . 9 
13.7 

II . 3 


Internet/Research Tools 


2001-2002 


88.9 


7.4 


1.9 


0.0 


1.9 


0.0 




2002-2003 


52.9 


3.9 


17.6 


5.9 


3.9 


5.9 




2003-2004 


75.5 


0.0 


3.8 


5.7 


9.4 


9.4 


Educational Software 


2001-2002 


79.6 


5.6 


1.9 


11.1 


5.6 


1.9 




2002-2003 


52.9 


5.9 


21.6 


9.8 


5.9 


0.0 




2003-2004 


56.6 


3.8 


17.0 


7.5 


1.9 


3.8 


Testing Software 


2001-2002 
















2002-2003 


68.6 


3.9 


5.9 


2.0 


2.0 


2.0 




2003-2004 


71.7 


0.0 


3.8 


3.8 


0.0 


1.9 


Notes. 1 ) Results marked as are not listed as they were reported in a different format or not collected in 2001 - 2002 . 
2 ) Item percentages may not total 100 % because of missing data. 
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Table 10 



A Summary of SCU Items Showing Significant Differences between Year 1 and Year 3 




Draw/Graphic 


0.02 


0.13 


0.52 


1.29 


3.85 


0.023 


+0.52 


Presentation 


0.00 


0.00 


0.27 


0.91 


5.87 


0.003 


+0.29 


Internet Browser 


0.07 


0.26 


0.33 


0.96 


4.47 


0.011 


+0.26 


Somewhat meaningful use of computers 


0.31 


0.88 


0.69 


1.38 


3.67 


0.027 


+0.30 


Meaningful use of computers 


0.11 


0.61 


0.68 


1.31 


14.95 


0.000 


+0.45 



Table 11 

SCU: Overall Meaningfulness of Computer Activities 



Year 1 (2001-2002) N = 54 
Year 2 (2002-2003) N = 51 
Year 3 (2003-2004) N = 53 




Low level use of computers 


2001-2002 

2002- 2003 

2003- 2004 


74.1 

68.6 

66.0 


1.9 

3.9 
9.4 


0.0 

3.9 

9.4 


0.0 

3.9 

7.5 


24.1 

5.9 

3.8 


Somewhat meaningful use of computers 


2001-2002 


81.5 


7.4 


3.7 


0.0 


3.7 




2002-2003 


54.9 


7.8 


5.9 


9.8 


9.8 




2003-2004 


73.6 


1.9 


3.8 


5.7 


9.4 


Meaningful use of computers 


2001-2002 


94.4 


0.0 


1.9 


0.0 


1.9 




2002-2003 


45.1 


2.0 


11.8 


9.8 


19.6 




2003-2004 


73.6 


3.8 


1.9 


11.3 


5.7 


Very meaningful use of computers 


2001-2002 


94.4 


0.0 


0.0 


0.0 


3.7 




2002-2003 


66.7 


3.9 


3.9 


0.0 


11.8 




2003-2004 


77.4 


1.9 


0.0 


1.9 


11.3 
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SCU: Year 1 vs. Year 3 Meaningful ness of Computer Use 
Frequently to Extensively Observed 
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24.1% 


3.7% 


1.9% 


3.7% 


■ Year 3 


11.3% 


15.1% 


17.0% 


13.2% 



Figure 3 



Summary. The SCU results suggest that progress was made between Year 1 and Year 3 with 
regard to technology integration efforts. Specifically, In Year 3 student more frequently used two 
production tools - draw/graphics and presentation - and more frequently used the Internet for 
research. In addition, the Year 3 data show that teachers were more capable of integrating student 
use of computers into their instruction in meaningful ways. 



SURVEY RESULTS 

Three surveys (TTQ, SCI, and TSA) were administered to the teachers during a faculty meeting 
held at each school in late May 2003. Results for the three instruments were examined for differences 
between years. 

School Climate Inventory (SCI) 

The SCI responses from 31 2 teachers remained fairly positive across Years 1 , 2, and 3, although 
there was a slight Year 1 to Year 3 decrease on five of the seven overall dimension mean scores and 
the overall mean (see Table 1 2). A slight shift also occurred when comparing the Year 3 findings to 
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national norms in that the data were favorable on four of the seven dimensions, but slightly lower on 
the remaining three (see Figure 4). Upon examining results by individual items, teachers reported the 
highest level of agreement for: “parents are treated courteously when they call or visit the school” 
(89.1%), “the administration communicates the belief that all students can learn” (88.8%), “teachers 
use a variety of teaching strategies or models” (88.5%), “the faculty and staff share a sense of 
commitment to the school goals” (86.5%), and “low achieving students are given opportunity for 
success in this school” (86.5%). The items having the lowest level of agreement were “students 
participate in solving the problems of the school” (30.1%), “parents are involved in home and school 
support network” (42.9%), and “teachers, administrators, and parents assume joint responsibility for 
student discipline” (43.3%). 
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Table 12 



School Climate Inventory Results 



Year 1 (2001-2002) N = 297 
Year 2 (2002-2003) N = 276 
Year 3 (2003-2004) N = 31 2 




COLLABORATION Items 



1 . The faculty and staff share a sense of commitment to the school goals. 


88.9 


93.1 


86.5 


6. Students are encouraged to help others with problems. 


74.4 


79.0 


70.5 


16. Teachers are encouraged to communicate concerns, questions, and constructive 


79.1 


85.5 


72.1 


ideas. 








26. Students participate in solving the problems of the school. 


25.9 


29.3 


30.1 


28. Faculty and staff cooperate a great deal in trying to achieve school goals. 


84.2 


89.5 


79.8 


31 . Teachers do not participate enough in decision making.** 


34.0 


27.5 


33.0 


40. Most problems facing this school can be solved by the principal and faculty. 


68.4 


65.6 


65.4 


ENVIRONMENT Items 








7. Faculty and staff feel that they make important contributions in this school. 


83.2 


86.6 


76.9 


9. Varied learning environments are provided to accommodate diverse teaching and 


81.5 


83.7 


79.8 


learning styles. 








10. The school building is neat, bright, clean, and comfortable. 


80.5 


72.1 


72.1 


14. School employees and students show respect for each other's individual differences. 


69.0 


67.8 


58.0 


29. An atmosphere of trust exists among the administration, faculty, staff, students, and 


57.6 


62.3 


48.1 


parents. 








38. Teachers are proud of this school and its students. 


85.5 


84.1 


70.2 


49. People in this school really care about each other. 


73.4 


78.3 


66.7 


EXPECTATIONS Items 








2. Low achieving students are given opportunity for success in this school. 


90.6 


94.6 


86.5 


3. School rules and expectations are clearly defined, stated, and communicated. 


81.5 


87.0 


82.1 


17. Students share the responsibility for keeping the school environment attractive and 


58.2 


54.7 


51.0 


clean. 








21 . Students are held responsible for their actions. 


66.0 


63.0 


65.7 


22. Many students in this school are not expected to master basic skills at each grade 


21.2 


19.9 


16.0 


level.** 








27. Many students do not participate in classroom activities because of their sex, race, 


14.1 


9.8 


8.7 


religion, socioeconomic status, or academic ability.** 








43. Teachers have high expectations for all students. 


81.1 


86.6 


82.4 


INSTRUCTION Items 








4. Teachers use a variety of teaching strategies or models. 


86.9 


94.2 


88.5 


15. Teachers sequence learning activities so that students can experience success at 


85.2 


87.7 


83.7 


each step. 








24. Teachers provide opportunities for students to develop higher-order skills. 


83.5 


90.6 


84.3 


33. Curriculum guides ensure that teachers cover similar subject content within each 


79.8 


86.2 


76.6 


grade level. 








35. Teachers use appropriate evaluation methods to determine student achievement. 


87.5 


91.7 


79.8 


41 . Pullout programs often disrupt and interfere with basic skills instruction.** 


26.9 


23.2 


21.8 


48. Teachers use a wide range of teaching materials and media. 


83.5 


92.0 


86.5 



**ltems are negatively worded, therefore a lower score = a more positive result. 
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Table 12 continued 







% Strongly Agree and Agree 


01-02 


02-03 1| 


03-04 


INVOLVEMENT Items 










5. Community businesses are active in this school. 




55.2 


54.0 


50.0 


1 1 . Parents are involved in a home and school support network. 




40.1 


41.3 


42.9 


12. Parents are treated courteously when they call or visit the school. 




92.6 


96.7 


89.1 


18. Parents are invited to serve on school advisory committees. 




77.1 


80.4 


79.8 


1 9. Parent volunteers are used wherever possible. 




72.7 


81.2 


72.4 


32. Information about school activities is communicated to parents on a consistent basis. 


78.5 


89.1 


80.4 


37. Parents are often invited to visit classrooms. 




57.6 


67.4 


63.1 


LEADERSHIP Items 










8. The administration communicates the belief that all students can learn. 




89.9 


92.8 


88.8 


20. The administration encourages teachers to be creative and to try new methods. 


82.5 


87.3 


81.7 


34. The administration provides useful feedback on staff performance. 




76.4 


87.0 


72.1 


36. The administrative staff does not do enough to protect instructional time.** 




17.8 


19.2 


19.9 


42. The principal is an effective instructional leader. 




74.1 


85.9 


74.4 


45. The goals of this school are reviewed and updated regularly. 




76.4 


77.2 


73.1 


47. The principal is highly visible throughout the school. 




74.7 


83.7 


81.1 


ORDER Items 










13. Rules for student behavior are consistently enforced. 




57.6 


65.2 


56.1 


23. Student discipline is administered fairly and appropriately. 




60.9 


57.2 


57.4 


25. Student misbehavior in this school interferes with the teaching.** 




54.5 


57.2 


67.9 


30. Student tardiness and absence from school is a major problem.** 




37.4 


38.4 


43.3 


39. The school is a safe and secure place in which to work. 




80.1 


78.6 


72.4 


44. Teachers, administrators, and parents assume joint responsibility for student 




52.5 


51.8 


43.3 


discipline. 










46. Student behavior is generally positive in this school. 




62.6 


60.1 


51.0 


** Items are negatively worded, therefore a lower score = a more positive result. 










SCI Section 2 










Dimensions by Overall Means and National Norms 










Dimension Year 1 Year 2 


Year 3 




National Norm 


Collaboration 3.71 3.79 


3.67 




3.74 




Environment* 3.94 3.87 


3.78 




3.79 




Expectations 3.88 3.90 


3.91 




3.85 




Instruction 3.99 4.08 


4.03 




4.02 




Involvement 3.78 3.93 


3.82 




3.77 




Leadership 3.96 4.08 


3.97 




3.92 




Order 3.35 3.27 


3.17 




3.22 




OVERALL 3.80 3.85 


3.76 




3.76 





*p<.01. 



Inferential Analysis. A MANOVA revealed significant differences existed between Years 1 , 2, and 
3; F(14,1316) = 5.09, p<. 001. Follow-up univariate analyses revealed that the Year 3 results for 
Environment were significantly ( p = .01 ) lower than the Year 1 results at an educationally important 
level (ES = -0.25). The Year 3 mean score was, however, only one percentage point below the 
national SCI norms (M= 3.79), indicating that the environment is similar to that of most US schools. 
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Year 1 vs. Year 3 SCI Results compared to National Norms 



4.5 n 






Instruction 
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Expectations 


Environment 
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Order 


□ Year 1 


3.99 
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3.22 



Figure 4 

Teacher Technology Questionnaire (TTQ) 

The Teacher Technology Questionnaire (TTQ) was designed to capture teacher perceptions 
regarding five areas: (1 ) impact of technology on classroom instruction, (2) impact of technology on 
students, (3) teacher readiness to integrate technology, (4) overall support for technology, and (5) 
technical support. A total of 207 teachers completed the survey. 

As seen Table 13, Year 3 as compared to Year 1 and 2 results show an increase in the overall 
percentage of teachers agreeing or strongly agreeing with items. The greatest Year 1 to Year 3 
change (27.8 percentage points) was for teacher “readiness to integrate technology” (Year 1 = 49.4%; 
Year 3 = 77.2%). Also noteworthy was a 20.3 percentage point increase in the number of Year 3 
teachers as compared to Year 1 teachers who were in agreement with the “impact on classroom 
instruction” items (Year 1 = 44.0%; Year 3 = 64.3%). In addition, the Year 3 demonstrated nearly 
75% or more overall agreement in teacher responses for all but one area (Impact on Classroom 
Instruction). 
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The most highly rated items on the survey were from “Impact on Students” and “Technical 
Support.” A majority of the teachers agreed that “most of the students can capably use computers at 
an age-appropriate level” (85.5%) and “most of the school computers are kept in good working 
condition” (85.0%). The lowest level of agreement for all three years was for “my teaching is more 
interactive when technology is integrated into the lessons” (Year 1 = 39.9%; Year 2 = 50.7%; Year 3 
= 59.4%). 
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Table 13 



Teacher Technology Questionnaire (TTQ) 



Year 1 (2001 -2002) N = 243* 
Year 2 (2002-2003) N = 268* 
Year 3 (2003-2004) N = 207 




Impact on Classroom Instruction 


01-02 02-03 


03-04 


01-02 02-03 03-04 


01-02 02-03 03-04 


14. 


My teaching is more student-centered when technology is 
integrated into the lessons. 


14.0 


14.9 


13.0 


49.0 


32.5 


22.7 


35.8 


51.1 


63.3 


16. 


1 routinely integrate the use of technology into my 
instruction. 


30.0 


18.7 


17.4 


28.8 


29.1 


20.8 


39.5 


50.4 


60.9 


18. 


Technology integration efforts have changed classroom 
learning activities in a very positive way. 


9.9 


6.0 


3.4 


28.8 


25.0 


22.2 


60.9 


67.2 


73.4 


20. 


My teaching is more interactive when technology is 
integrated into the lessons. 


14.8 


18.3 


15.9 


44.4 


29.1 


22.7 


39.9 


50.7 


59.4 




Overall 


17.2 


14.5 


12.4 


37.8 


28.9 


22.1 


44.0 


54.9 


64.3 


Impact on Students 




















3. 


The use of computers has increased the level of student 
interaction and/or collaboration. 


9.9 


10.8 


4.3 


26.3 


20.1 


16.9 


63.4 


67.9 


76.8 


8. 


The integration of technology has positively impacted 
student learning and achievement. 


9.1 


7.1 


8.7 


24.3 


19.4 


11.1 


65.0 


71.6 


79.2 


10 . 


Most of my students can capably use computers at an age- 
appropriate level. 


7.0 


7.5 


6.8 


18.5 


15.7 


6.8 


73.7 


75.7 


85.5 


19. 


The use of technology has improved the quality of student 
work. 


11.9 


11.9 


11.1 


30.0 


32.5 


29.5 


58.0 


53.7 


58.5 




Overall 


9.5 


9.3 


7.7 


24.8 


21.9 


16.1 


65.0 


67.2 


75.0 


Teacher Readiness to Integrate Technology 




















5. 


1 know how to meaningfully integrate technology into 
lessons. 


17.3 


9.0 


10.1 


31.7 


17.9 


7.2 


50.2 


71.6 


81.2 


9. 


1 am able to align technology use with my district's 
standards-based curriculum. 


14.0 


9.3 


9.2 


38.3 


22.8 


20.3 


46.1 


65.7 


68.6 


11. 


1 have received adequate training to incorporate technology 
into my instruction. 


25.5 


10.1 


10.1 


27.6 


16.4 


11.6 


46.1 


72.8 


77.8 


12. 


My computer skills are adequate to conduct classes that 
have students using technology. 


18.5 


9.7 


9.7 


25.5 


13.1 


8.2 


55.1 


75.7 


81.2 




Overall 


18.8 


9.5 


9.8 


30.8 


17.6 


11.8 


49.4 


71.5 


77.2 


Overall Support for Technology in the School 




















4. 


Parents and community members support our school's 
emphasis on technology. 


6.2 


6.3 


1.4 


29.2 


30.2 


27.5 


64.2 


61.2 


70.0 


13. 


Teachers receive adequate administrative support to 
integrate technology into classroom practices. 


14.0 


6.0 


6.3 


28.4 


15.7 


13.5 


57.2 


76.9 


79.7 


15. 


Our school has a well-developed technology plan that 
guides all technology integration efforts. 


17.3 


11.9 


9.7 


33.3 


33.2 


27.1 


49.0 


52.6 


62.8 


17. 


Teachers in this school are generally supportive of 
technology integration efforts. 


7.4 


6.3 


4.3 


19.3 


17.9 


14.0 


72.8 


75.0 


80.2 




Overall 


11.2 


7.6 


5.4 


27.6 


24.3 


20.5 


60.8 


66.4 


73.2 


Technical Support 




















1 . 


Most of our school computers are kept in good working 
condition. 


9.1 


9.3 


7.2 


9.9 


10.4 


6.8 


80.2 


79.5 


85.0 


2. 


1 can readily obtain answers to technology-related 
questions. 


12.8 


6.3 


7.7 


14.8 


11.6 


10.1 


71.6 


81.0 


81.2 


6. 


My students have adequate access to up-to-date 
technology resources. 


19.8 


17.2 


14.5 


24.7 


17.5 


15.9 


52.7 


62.3 


69.1 


7. 


Materials ( e.g ., software, printer supplies) for classroom use 
of computers are readily available. 


23.0 


22.0 


23.2 


21.4 


14.9 


14.0 


55.6 


61.2 


62.3 




Overall 


16.2 


13.7 


13.2 


17.7 


13.6 


11.7 


65.0 


71.0 


74.4 



*Note: Item percentages may not total 1 00% because of missing input from some respondents. 
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Table 13 continued 

TTQ Section 2: Participant Information 

Percentages by Categories 

Note: Percentages may not total 100% because of missing input from some respondents. 




How would you rate your level of computer ability? 


Very Good 


16.9 


16.4 


18.8 




Good 


25.5 


41.4 


51.7 




Moderate 


45.7 


34.7 


26.1 




Poor 


11.5 


6.3 


1.4 




No Ability 


0.0 


0.0 


0.0 


Do you own a home computer? 


Yes 


89.7 


89.2 


92.8 




No 


9.1 


10.1 


5.3 


If yes, do you use your home computer to access 


Yes 


71.6 


72.8 


80.7 


instructional materials on the Internet? 


No 


26.1 


19.7 


15.6 


If yes, do you use your home computer to prepare 


Yes 


66.1 


69.5 


81.3 


instructional materials? 


No 


30.3 


21.8 


14.1 



Inferential results: To determine if significant differences existed between Years 1 , 2, and 3, 
a MANOVA and follow-up univariate analyses were conducted to compare the three sets of data. 
Table 14 presents a summary of the results and reveals a significant multivariate effect, F(10, 

1 300) = 7.51 , p < .001 . Follow-up analyses revealed significant differences in each of the five 
TTQ areas. A significantly higher level of agreement among teachers was demonstrated for 
Years 2 and 3 when each was compared to the Year 1 baseline in the following areas: “teacher 
readiness to integrate technology” (p = .001 ); “overall support” (p = .001 )\ and “impact on 
classroom instruction” (p = .001 ). A significantly higher agreement was also indicated for Year 3 
when compared to Year 2 for “impact on classroom instruction”. Regarding “impact on students”, 
significantly higher agreement was revealed for Year 3 when compared to both the Year 1 
baseline and Year 2 (p = .001 ). Significantly higher agreement was indicated for Year 3 over the 
Year 1 baseline for the area of Technical Support, (p = .004). Figure 5 depicts Year 1 to Year 3 
comparisons. 
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Table 14 



TTQ: Significant Differences Between Program Years 



Overall 



Hotellings T 



0.12 



Hypothesis df 



Error df Significance 



7.51 



10.00 



1296.00 



0.000 



2003-2004 




Teacher Readiness to Integrate 
Technology 


3.40 


0.80 


3.85 


0.76 


3.96 


0.77 


30.86 


0.000 


+0.73 


Impact on Classroom Instruction 


3.32 


0.75 


3.53 


0.78 


3.71 


0.79 


12.98 


0.000 


+0.51 


Impact on Students 


3.69 


0.65 


3.76 


0.70 


3.92 


0.70 


5.85 


0.003 


+0.34 


Overall Support for Technology in the 
School 


3.60 


0.62 


3.77 


0.63 


3.90 


0.65 


11.61 


0.000 


+0.48 


Technical Support 


3.61 


0.76 


3.77 


0.75 


3.83 


0.74 


5.06 


0.007 


+0.29 



TTQ: Year 1 and Year 3 Results 



5.00 

4.00 

3.00 

2.00 
1.00 






Readiness to 
Integrate Tech (ES 
+0.73) 


Impact on 
Instruction (ES 
+0.51) 


Impact on Students 
(ES +0.34) 


Overall Support for 
Tech (ES +0.48) 


Technical Support 
(ES +0.29) 


□ Year 1 


3.40 


3.32 


3.69 


3.60 


3.61 


■ Year 3 


3.96 


3.71 


3.92 


3.90 


3.83 



Figure 5 
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Technology Skills Assessment (TSA) 

The primary purpose for the TSA was to assess teacher perceptions of their technology ability as 
to tasks noted in the TEKS Technology Applications for grade 3-5 students. The survey begins by 
asking the teachers to rate “How easily. . .” (Not at all, Somewhat, Very easily) they could use or 
complete 47 computer-related tasks divided into six basic areas: (1) computers, (2) software, (3) 
multimedia, (4) Internet, (5) advanced skills, and (6) using technology for learning. The teachers were 
also asked to rate “How well. . .” (Not at all, Somewhat, Very well) they understood three technology- 
related policy and ethics items. A summary of the results is presented in Table 1 5. 

Year 3 results showed positive increases in teacher confidence in completing technology-related 
tasks, specifically related to “Internet basics” (Year 1 = 24.5%; Year 3 = 49.8%), “software basics” 
(Year 1 = 45.2%; Year 3 = 66.5%), and “using technology for learning” (Year 1=11 .3%; Year 3 = 
30.4%). Only one area showed a slight Year 1 to Year 3 decrease (2 percentage points) in teacher 
confidence, “multimedia basics” (Year 1 = 48.4%; Year 3 = 46.4%). This should not be of concern, 
however, since almost half of the teachers report that they can “very easily” complete tasks that 
require multimedia software. 

Year 3 results also show that a majority of the teachers feel they can easily complete basic 
“computer” (84.8%) tasks, and over two-thirds expressed confidence that they can easily complete 
basic “software” (66.5%) tasks and have a clear understanding of basic technology-related “policy and 
ethics” (69.3%) issues. Only a little less than one-third of the teachers felt confident in being able to 
“very easily” complete “advanced” (30.0%) tasks and use “ technology for learning” (30.4%). 
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Table 15 



Technology Skills Assessment Data Summary 




Computer Basics ~ How easily can you ... 


01-02 02-03 03-04 


01-02 02-03 03-04 


01-02 02-03 03-04 


Use a spell check tool. 


5.6 


2.9 


0.5 


14.0 


8.1 


4.3 


80.0 


88.6 


94.7 


Create basic computer documents (word processed) in a timely manner. 


6.0 


3.3 


0.0 


23.9 


14.0 


10.6 


69.5 


82.0 


88.9 


Use help menus for software programs. 


6.3 


3.3 


2.4 


38.6 


21.7 


16.9 


53.7 


74.3 


77.8 


Use basic computer terms like mouse, keyboard, hard drive, CD-ROM, 
and monitor. 


1.8 


0.0 


0.0 


15.8 


7.7 


4.8 


81.4 


91.9 


92.8 


Save documents so they can be opened on both a Macintosh and PC. 


20.7 


11.4 


4.8 


29.1 


25.4 


22.7 


49.1 


62.1 


71.0 


Create folders on a hard drive or disk. 


16.1 


7.4 


3.4 


35.1 


18.8 


21.3 


47.4 


73.2 


74.9 


Save files to specific folders. 


13.3 


4.8 


1.0 


33.0 


22.1 


18.4 


52.6 


73.2 


80.2 


Locate and delete unwanted files. 


7.7 


2.6 


1.0 


25.6 


19.1 


13.0 


65.3 


77.2 


85.5 


Use keyboard commands to cut, copy, or delete text. 


11.2 


5.5 


3.9 


28.8 


20.2 


14.5 


59.6 


74.3 


79.7 


Proficiently use a mouse and keyboard. 


1.4 


0.0 


0.0 


15.8 


8.5 


8.7 


81.8 


91.5 


91.3 


Print a document using "Print" from the File menu and/or the toolbar 


2.1 


1.1 


0.5 


9.1 


4.4 


3.4 


88.1 


93.4 


95.7 


icon. 




















Computer Basics: Overall 


8.4 


3.8 


1.6 


24.4 


15.5 


12.6 


66.2 


80.2 


84.8 


Software Basics 




















Use software preview features to check work. 


17.2 


7.7 


4.3 


30.5 


22.8 


21.7 


51.6 


68.8 


72.5 


Open and use software programs that are installed on your computer. 


3.5 


1.5 


1.0 


29.5 


18.4 


10.6 


66.0 


79.4 


87.4 


Work with and move between two open programs (e.g., Internet and 
database) to create a product. 


30.9 


11.8 


7.2 


30.9 


31.6 


27.1 


38.2 


56.3 


65.2 


Describe the difference between downloading and installing software. 


21.4 


10.3 


6.3 


36.1 


30.5 


31.4 


42.1 


58.5 


61.8 


Save documents so they can be opened in a different program (e.g., 
from Word to Word Perfect). 


33.7 


14.7 


10.1 


32.6 


34.6 


31.4 


33.0 


49.6 


57.0 


Install software. 


26.3 


14.7 


7.7 


30.2 


26.8 


30.4 


40.0 


53.7 


55.1 


Software Basics: Overall 


22.2 


10.1 


6.1 


31.6 


27.5 


25.4 


45.2 


61.1 


66.5 


Multimedia Basics 




















Import digital video from a camera to a computer. 


62.1 


47.4 


37.7 


21.1 


26.1 


30.4 


16.5 


25.7 


30.4 


Record and save your voice onto a computer. 


72.6 


57.7 


48.3 


18.9 


22.8 


25.6 


8.1 


19.5 


24.2 


Use a scanner to import a photo or document into a computer. 


49.8 


39.7 


31.4 


25.6 


23.9 


20.8 


23.9 


35.7 


45.9 


Play a music CD on the computer. 


11.2 


7.4 


3.9 


22.1 


13.6 


10.6 


66.0 


78.7 


85.0 


Multimedia Basics: Overall 


24.7 


38.1 


30.3 


26.1 


21.6 


21.9 


48.4 


39.9 


46.4 


Internet Basics 




















Connect to the Internet with a modem (phone, cable). 


18.2 


11.8 


7.2 


24.2 


11.8 


8.2 


57.2 


76.1 


83.6 


Use Boolean strategies for Internet searches. 


62.1 


39.0 


23.2 


20.0 


21.7 


26.1 


15.8 


37.1 


47.3 


Use appropriate software and the Internet to find audio, video, and 
graphics for lesson plans. 


38.2 


16.5 


11.6 


36.5 


37.5 


35.7 


24.6 


44.9 


52.2 


Use the Internet to find help when you have a computer problem. 


43.9 


26.5 


20.3 


33.7 


35.7 


33.8 


22.1 


36.4 


44.0 


Determine if information you find on the Internet is accurate and valid. 


40.0 


23.9 


15.9 


41.1 


39.7 


44.9 


18.9 


35.7 


37.7 


Evaluate Internet search strategies to determine those that are most 
efficient. 


41.4 


24.3 


15.0 


38.2 


35.3 


37.7 


20.4 


39.3 


46.4 


Determine the usefulness and appropriateness of digital information. 


54.7 


33.8 


25.1 


32.3 


33.1 


36.2 


12.3 


32.0 


37.7 


Internet Basics: Overall 


42.6 


25.1 


16.9 


32.3 


30.7 


31.8 


24.5 


43.1 


49.8 
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Table 15 continued 




Advanced Skills 


01-02 02-03 03-04 


01-02 02-03 03-04 


01-02 02-03 03-04 


Use more advanced computer terms like megahertz, gigabytes, and 
RAM. 


53.0 


38.6 


30.0 


31.2 


39.3 


41.1 


14.4 


21.0 


26.1 


Access information on local area networks (LANs) and wide area 
networks (WANs). 


60.0 


39.0 


33.3 


26.3 


34.2 


35.7 


11.6 


26.1 


27.1 


Use appropriate digital layout and design to meet the needs of defined 
audiences. 


68.1 


47.8 


38.6 


20.7 


32.0 


31.9 


8.4 


18.8 


25.6 


Use appropriate digital layout and design for the selected media (e.g., 
multimedia, web, print). 


67.0 


45.6 


33.8 


22.8 


32.0 


38.6 


8.4 


21.3 


23.7 


Publish information in a variety of media (e.g., printed, monitor display, 
web-based, video). 


61.1 


38.6 


29.0 


28.1 


35.3 


41.1 


8.4 


23.9 


25.6 


Connect a computer to a local server to share files. 


67.0 


47.1 


37.2 


23.9 


26.8 


34.8 


7.7 


24.6 


22.7 


Determine if a software program works with an operating system. 


64.9 


48.5 


38.2 


23.5 


26.5 


30.9 


9.1 


23.2 


25.1 


Print to a specific printer when connected to a network that has more 
than one printer. 


46.0 


28.7 


19.3 


28.8 


26.1 


24.6 


23.5 


44.1 


52.7 


Use presentation software to share information with specific audiences. 


58.6 


30.1 


21.7 


24.6 


32.0 


32.9 


14.4 


34.9 


41.1 


Advanced Skills: Overall 

Using Technology for Learning 


60.6 


40.4 


31.2 


25.5 


31.6 


34.6 


11.8 


26.4 


30.0 


Use multimedia software to enhance learning experiences. 


42.8 


18.8 


13.5 


38.6 


42.6 


42.5 


16.5 


36.4 


40.6 


Use appropriate software (e.g., word processing, graphics, databases, 
spreadsheets, simulations, and multimedia) to express ideas and solve 
problems. 


38.2 


16.5 


10.1 


36.8 


43.8 


40.6 


22.8 


38.2 


45.9 


Use text and graphics to create and modify solutions to problems. 


52.6 


26.1 


20.3 


28.8 


41.9 


39.6 


16.8 


30.9 


37.7 


Use digital audio and video to create and modify solutions to problems. 


70.2 


44.1 


35.3 


21.8 


37.9 


38.2 


6.3 


16.5 


22.2 


Use communication tools to participate in group projects. 


60.0 


33.5 


25.1 


24.9 


39.0 


39.6 


11.9 


25.4 


31.4 


Manipulate information in interactive digital environments (e.g., 
simulations, virtual labs, field trips). 


71.6 


50.4 


44.4 


20.0 


30.5 


30.9 


6.0 


17.6 


20.3 


Participate in a listserv, chat, and bulletin board session. 


62.1 


41.2 


27.5 


23.9 


25.4 


33.8 


11.9 


32.0 


34.8 


Create an electronic teaching portfolio to evaluate your work. 


74.7 


52.2 


41.1 


17.9 


29.8 


37.2 


5.3 


16.2 


18.8 


Evaluate electronic portfolio products. 


74.7 


52.2 


44.4 


18.9 


30.9 


35.3 


3.5 


14.7 


17.4 


Create technology tools to assess student work (e.g., checklists, 
timelines, rubrics). 


58.9 


31.3 


27.5 


26.3 


37.9 


33.3 


11.6 


28.7 


35.3 


Using Technology for Learning: Overall 

Policy and Ethics 


60.6 


36.6 


28.9 


25.8 


36.0 


28.8 


11.3 


25.7 


30.4 


My school's acceptable use policy. 


6.0 


1.8 


1.9 


25.3 


18.0 


14.5 


64.6 


76.8 


81.2 


The concept of a school site license for software. 


9.5 


5.9 


1.4 


30.2 


24.3 


26.1 


56.5 


66.2 


69.6 


How to determine if it is legal to copy a software program or another 
individual's electronic work. 


18.6 


9.9 


4.8 


38.2 


34.2 


35.3 


38.6 


52.2 


57.0 


Policy and Ethics: Overall 


11.4 


5.9 


2.7 


31.2 


25.5 


25.3 


53.2 


65.1 


69.3 



Inferential results. A MANOVA comparing the means of Years 1 , 2 and 3 on the seven TSA 
categories (computer, software, multimedia, Internet, advanced skills, using technology for learning, 
and policy and ethics) yielded a highly significant difference, F(14, 1110) = 6.58, p < .0001. As seen in 
Table 1 6, follow-up analyses revealed significant differences between years on all dimensions. Most 
notable is the Year 1 (M= 1 .52) to Year 3{M= 2.07) increase in teacher confidence to use technology 
for learning, which had an impressive Effect Size of +0.90. Figure 6 highlights the differences for 
Year 1 and 3. 
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Table 16 



TSA: Significant Differences Between Years 



Overall Hotellings T 




F 






Hypothesis df 




Error df 




Significance 


0.15 




6.58 




14.00 




1106.00 




0.000 












Using Technology 


1.52 


0.56 


1.92 


0.67 


2.07 


0.62 


48.15 


.000 


+0.90 


Internet Basics 


1.86 


0.62 


2.23 


0.62 


2.38 


0.59 


47.90 


.000 


+0.82 


Advanced Skills 


1.54 


0.61 


1.88 


0.69 


2.03 


0.65 


38.51 


.000 


+0.75 


Multimedia Basics 


1.81 


0.56 


2.02 


0.64 


2.23 


0.57 


23.56 


.000 


+0.73 


Software Basics 


2.28 


0.60 


2.53 


0.55 


2.63 


0.45 


30.87 


.000 


+0.65 


Computer Basics 


2.62 


0.41 


2.63 


0.37 


2.85 


0.26 


29.69 


.000 


+0.64 


Policy and Ethics 


2.44 


0.58 


2.62 


0.51 


2.70 


0.44 


13.75 


.000 


+0.50 



* Sorted from highest to lowest Effect Size 



TSA: Year 1 and Year 3 Results 



Computer Software Multimedia . . , D Advanced ... _ . ,-. .. 0 

Basics (ES Basics (ES Basics (ES '“ n rT Skills (ES ^ ^ 

+0.64) +0.65) +0.73) (ES +082) +Q 75) (ES +0.90) (ES +0.50) 

2.62 2.28 1.81 1.86 1.54 1.52 2.44 

2.85 2.63 2.23 2.38 2.03 2.07 2.7 



Figure 6 
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CONCLUSIONS 

The conclusions of the study will be presented in association with each of the major research 
questions in the respective sections below. 

To what degree and how is technology integrated with classroom instruction by teachers in 
general* and those who have received Foundation and/or integration training through the 
Enhanced Learning Academy? 

The observation data revealed significant Year 1 to Year 3 increases in teacher use of 
computers for instructional delivery and in student use of computers as a learning tool. Specifically, in 
nearly three-quarters (72.2%) of the observations technology was used as a means of delivering 
instruction through the use of educational software. Also impressive was student use of technology 
as a tool to enhance learning, which was seen in nearly two-thirds (61 .1%) of the visits. Another 
noteworthy increase in the use of computers is seen in data from the RSCA, which revealed that 
during Year 3 as compared to Year 1 , teachers more frequently supported student-centered activities 
with technology. This was demonstrated in over 30% of the observations, in which computers were 
used to support “experiential hands-on learning activities” (31 .5%) and the use of “higher level 
questioning strategies” (38.9%). Not only were teachers more frequently integrating technology into 
their instruction, they were able to use a greater variety of software (Year 1 = 6 of 14 software 
applications; Year 3 = 11 of 14) and able to do so at a significantly more meaningful level. 

Specifically, during Year 1 “somewhat meaningful” to “meaningful” computer activities were only 
“frequently” to “extensively” observed in 5.6% of the classroom visits as compared to 32.1 % of the 
Year 3 visits. Improvements occurred in the frequency, variety and quality of technology integration 
activities, and significant increases in the types of computer activities were observed with respect to 
three applications: draw/graphics, presentation, and Internet browsers. These results are highly 
suggestive that the STARK Program is effectively impacting technology integration efforts in the target 
schools. Continued professional development is needed, however, to expand teacher ability to 
engage students in use of technology as a learning tool. 

To what degree do integration-trained teachers use methodologies that, consistent with 
STARK Program goals, stress higher-order learning and student-centered learning activities? 

The Year 3 results revealed consistent and often significant increases in the frequency with 
which activities that foster critical thinking and student engagement were observed. Most striking 
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were significant increases in the percent of visits in which the teachers utilized “higher-order 
questioning” (Year 1 = 35.2%; Year 3 = 59.3%) and had students use “technology as a learning tool” 
(Year 1 = 18.5%; Year 3 = 61 .1%). Although the differences were not statistically significant, Year 3 
teachers more frequently assumed the “role of a facilitator” (Year 1 = 59.3%; Year 3 = 68.5%) and 
engaged students in discussions (Year 1 = 29.6%; Year 3 = 38.9%) while continuing to provide 
“experiential hands-on activities” (Year 1 = 42.6%; Year 3 = 44.4%). Although no significant increases 
were found in the quality with which teachers implement student-centered activities, it is promising to 
note that all Year 3 RSCA mean scores were above average - with five being higher than Year 1 
ratings. Of concern, though was the limited use of “independent inquiry/research on the part of the 
students” which was only seen in 1 1 .1% of the observations for both Years 1 and 3. The data indicate 
that teachers are better able to create classroom environments that engage students in higher-order 
thinking and learning. Professional development efforts should, however, increase their focus on 
better preparing teachers to implement these methods to increase the frequency with which they are 
utilized. 

To what degree have teachers acquired the technology skills specified in the Texas State 
standards? 

Impressive Year 3 TSA results showed that teacher confidence on all seven categories of 
technology skills continued to significantly increase as compared to Year 1 and 2. The greatest 
difference (ES = +0.90) was seen in teacher confidence in using “technology for learning.” Between 
Year 1 and Year 3, the percentage of teachers who felt they could “very easily” implement this 
strategy increased from 1 1 .1% to nearly one-third (30.4%). The majority of the teachers expressed 
confidence that they could very easily complete basic computer and software tasks and that they 
understood the policy and ethics of technology use, and nearly half felt very confident when using the 
Internet and multimedia software. The percent of teachers who indicated they could very easily 
complete advanced tasks increased to nearly the same degree as the percent of those who could use 
technology for learning (Year 1 = 11 .8%; Year 3 = 30.0%). These results are a strong indication of 
successful teacher training through the STARK Program. 
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What are school outcomes in school climate and teacher uses of and attitudes toward 
technology? To what extent do these variables (a) reflect Foundation training and STARK 
Program goals, and (b) correlate with one another and with implementation success? 

Results from the Year 3 School Climate Inventory were mixed, with four mean scores being 
higher than Year 1 and three being lower - only one to a significant degree (Environment - Year 1 M = 
3.94; Year 3 M = 3.78). Even with lower SCI ratings, the environment of the participating schools was 
still positive enough to support significant improvements in school-wide technology integration efforts. 
This was evidenced in the significant increases in frequency of classroom use of technology and 
teacher ability to create and implement meaningful computer activities. Positive teacher attitudes 
were also reflected in the TTQ results, which showed significantly higher teacher readiness to 
integrate technology and more confidence to conduct classes that have students use technology. 
Additionally, there was nearly unanimous teacher agreement that their district/schools provided 
necessary technical and overall support for the technology program. These findings suggest that 
although slight shifts may occur in school climate, the district is supportive of the technology initiative 
and the STARK Program had a positive impact on teacher readiness to integrate technology and 
teacher belief that technology has a positive impact on classroom instruction. 



What factors appear most instrumental in determining schools’ success at achieving the goals 
and overall implementation of the STARK Program? 

Below is a list of key factors from the Year 3 study that appeared to have influenced the progress 
being made toward achievement of the STARK Program implementation and goals. As seen, the 
factors address key elements that are critical for program success: 



Significant differences in instructional practices 

• Increased use of student-centered learning 

• Increased use of computers for instructional delivery 

• Increased use of technology as a learning tool 

Significant differences in the quality of instructional practices 

• Increased use of meaningful computer activities 

Significant differences in teacher attitudes and beliefs regarding technology integration 

• The majority of teachers felt ready to integrate technology 

• The majority of teachers believed technology positively impacts classroom instruction 

• The majority of teachers agreed technology efforts are well-supported 

Significant differences in teacher computer skills 

• The majority of teachers reported confidence with technology tasks 
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In general, the Year 3 results reflect impressive progress toward achieving the STARK Program 
goals. Specifically, this year’s data revealed significant changes from Year 1 in that teachers were 
more technologically competent, more frequently integrated technology into their instruction, and more 
frequently engaged students in meaningful technology-supported activities. Even though significant 
strides have been made, the scope of technology integration was somewhat limited with regard to the 
variety of software used and the overall degree of computer use. In order to enhance student 
experiences and maximize opportunities for improved learning, program expansion is recommended. 
Suggested areas of expansion include: (1) purchasing more computers to equalize distribution of 
access to all students; (2) implementing a computer maintenance and replacement program to ensure 
up-to-date computers are available for student use; (3) increasing teacher support through the use of 
onsite Technology Coaches who provide personalized, just-in-time training; and (4) continuing and 
expanding current professional development to build teacher capacity to fully utilize technology 
resources. 
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Appendix A 



Alignment of Technology Skills Assessment with NETS and TEKS 
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Alignment of Technology Skills Assessment with NETS and TEKS 



NETS for Teachers 

I. TECHNOLOGY OPERATIONS AND CONCEPTS: Teachers demonstrate a sound understanding of technology operations and concepts. Teachers: 

A. demonstrate introductory knowledge, skills, and understanding of concepts related to technology (as described in the ISTE National Education Technology Standards for Students). 



NETS for Students 
Technology Foundation 
Standards for All Students 


NETS for Students 
Grades 3-5 


TEKS Technology Applications 
Grades 3-5 


Technology Skills Assessment 


http://cnets.iste.org/index2.html 


http://cnets.iste.org/index2.html 


http://www.tea.state.tx.us/rules/tac/ch1 26.html#s1 263 




1 ) Basic operations and concepts 

• Students demonstrate a 
sound understanding of the 
nature and operation of 
technology systems. 

• Students are proficient in the 
use of technology. 


1 . Use keyboards and other common input 
and output devices (including adaptive 
devices when necessary) efficiently and 
effectively. (1) 

2. Discuss common uses of technology in 
daily life and the advantages and 
disadvantages those uses provide. (1 , 2) 


1 . Foundations. The student demonstrates 
knowledge and appropriate use of hardware 
components, software programs, and their 
connections. The student is expected to: 




1 . use technology terminology appropriate to 
the task; 


4. use basic computer terms like mouse, 
keyboard, hard drive, CD-ROM and 
monitor. 

29. use more advanced computer terms 
like megahertz, gigabytes, and RAM. 

55. If you were asked to describe the size 
of a hard drive, which measure would 
be most appropriate to use in your 
description? 


2. save and delete files, uses menu options 
and commands, and work with more than 
one software application; 


7. save files to specific folders. 

8. locate and delete unwanted files. 

9. use keyboard commands to cut, copy 
or delete text. 

1 1 . print a document using “Print” from the 
File menu and/or the toolbar icon. 

1 3. open and use software programs that 
are installed on my computer. 

1 4. work with and move between two 
open programs (e.g. internet and 
database) to create a product. 

51 . Under which menu item is the “New” 
document option typically found? 

53. Which of the following is a format 
used to save graphics? (see also 1 -C) 
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3. identify and describe the characteristics of digital 
input, processing, and output; 



4. delineate and make necessary adjustments 
regarding compatibility issues including, but not 
limited to, digital file formats and cross platform 
connectivity; and 



5. access remote equipment on a network such as a 
printer or other peripherals. 



2. Foundations. The student uses data input skills 
appropriate to the task. The student is expected to: 

2. use a variety of input devices such as mouse, 
keyboard, disk drive, modem, voice/sound 
recorder, scanner, digital video, CD-ROM, or touch 
screen; 



6. create folders on a hard drive or disk 
(see also 2-A) 

1 5. describe the difference between 
downloading and installing software. 

1 8. import digital video from a camera to a 
computer, (see also 2-A) 

20. use a scanner to import a photo or 
document into a computer, (see also 
2-A) 

53. Which of the following is a format 
used to save graphics? (see also 1 -C) 

5. save documents so they can be 
opened on both a Macintosh and PC. 

1 6. save documents so they can be 
opened in a different program (e.g. 
from Word to Word Perfect). 

35. determine if a software program works 
with an operating system. 

52. How should a Macintosh-created 
word processed document be named 
so it can be opened on a PC? 

34. connect a computer to a local server 
to share files. 

36. print to a specific printer when 

connected to a network that has more 
than one printer. 



6. create folders on a hard drive or disk 
(see also 1 -C) 

1 0. proficiently use a mouse and 
keyboard, (see also 2-B, 2-C, 2-D) 

17. install software. 

1 8. import digital video from a camera to a 
computer, (see also 1 -C) 

1 9. record and save my voice onto a 
computer. 

20. use a scanner to import a photo or 
document into a computer, (see also 
1-C) 

21 . play a music CD on my computer. 

22. connect to the Internet with a modem 

(phone, cable). 
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3. use proper keyboarding techniques such as 
correct hand and body positions and smooth and 
rhythmic keystroke patterns; 


1 0. proficiently use a mouse and 
keyboard, (see also 2-A, 2-C, 2-D) 






4. demonstrate touch keyboarding techniques for 
operating the alphabetic, numeric, punctuation, 
and symbol keys as grade-level appropriate; 


1 0. proficiently use a mouse and 
keyboard, (see also 2-A, 2-B, 2-D) 






5. produce documents at the keyboard, proofread, 
and correct errors; 


1 . use a spell check tool, (see also 2-E) 

2. create basic computer documents 

(word processed) in a timely manner, 
(see also 2-F) 

1 0. proficiently use a mouse and 
keyboard, (see also 2-A, 2-B, 2-C) 






6. use language skills including capitalization, 
punctuation, spelling, word division, and use of 
numbers and symbols as grade-level appropriate; 
and 


1 . use a spell check tool, (see also 2-D) 






7. demonstrate an appropriate speed on short timed 
exercises depending upon the grade level and 
hours of instruction 


2. create basic computer documents 
(word processed) in a timely manner, 
(see also 2-D) 


2) Social, ethical, and human issues 
• Students understand the 
ethical, cultural, and societal 


2. Discuss common uses of 
technology in daily life and the 
advantages and disadvantages those 
uses provide. (1 , 2) 


3. Foundations. The student complies with the laws and 
examines the issues regarding the use of technology in 
society. The student is expected to: 




issues related to technology. 
• Students practice 


A) follow acceptable use policies when using 
computers; and 


48. my school's acceptable use policy. 


responsible use of 
technology systems, 
information, and software. 

• Students develop positive 
attitudes toward technology 
uses that support lifelong 
learning, collaboration, 
personal pursuits, and 
productivity. 


3. Discuss basic issues related to 
responsible use of technology and 
information and describe personal 
consequences of inappropriate use. 
(2) 


B) model respect of intellectual property by not 
illegally copying software or another individual’s 
electronic work. 


49. the concept of a school site license for 
software. 

50. how to determine if it is legal to copy a 
software program or another 
individual’s electronic work. 
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3) Technology productivity tools 

• Students use technology 
tools to enhance learning, 
increase productivity, and 
promote creativity. 

• Students use productivity 
tools to collaborate in 
constructing technology- 
enhanced models, prepare 
publications, and produce 
other creative works. 



4) Technology communications tools 

• Students use 
telecommunications to 
collaborate, publish, and 
interact with peers, experts, 
and other audiences. 

• Students use a variety of 
media and formats to 
communicate information 
and ideas effectively to 
multiple audiences. 



4. Use general purpose productivity 
tools and peripherals to support 
personal productivity, remediate skill 
deficits, and facilitate learning 
throughout the curriculum. (3) 

5. Use technology tools (e.g., 
multimedia authoring, presentation, 
Web tools, digital cameras, scanners) 
for individual and collaborative writing, 
communication, and publishing 
activities to create knowledge 
products for audiences inside and 
outside the classroom. (3, 4) 

5. Use technology tools (e.g., 
multimedia authoring, presentation, 
Web tools, digital cameras, scanners) 
for individual and collaborative writing, 
communication, and publishing 
activities to create knowledge 
products for audiences inside and 
outside the classroom. (3, 4) 

6. Use telecommunications efficiently 
to access remote information, 
communicate with others in support of 
direct and independent learning, and 
pursue personal interests. (4) 



7. Use telecommunications and online 
resources (e.g., e-mail, online 
discussions, Web environments) to 
participate in collaborative problem- 
solving activities for the purpose of 
developing solutions or products for 
audiences inside and outside the 
classroom. (4, 5) 



7. Solving problems. The student uses appropriate 
computer-based productivity tools to create and modify 
solutions to problems. The student is expected to: 




A) use software programs with audio, video, and 
graphics to enhance learning experiences; 


38. use multimedia software to enhance 
learning experiences. 


B) use appropriate software to express ideas and 
solve problems including the use of word 
processing, graphics, databases, spreadsheets, 
simulations, and multimedia; and 


39. use appropriate software (e.g. word 
processing, graphics, databases, 
spreadsheets, simulations, and 
multimedia) to express ideas and 
solve problems. 


C) use a variety of data types including text, graphics, 
digital audio, and video. 


40. use text and graphics to create and 
modify solutions to problems. 

41 . use digital audio and video to create 
and modify solutions to problems. 


10. Communication. The student formats digital information 
for appropriate and effective communication. The 
student is expected to: 




A) use font attributes, color, white space, and graphics 
to ensure that products are appropriate for the 
defined audience; 


31 . use appropriate digital layout and 
design to meet the needs of defined 
audiences. 


B) use font attributes, color, white space, and graphics 
to ensure that products are appropriate for the 
communication media including multimedia screen 
displays, Internet documents, and printed 
materials; and 


32. use appropriate digital layout and 
design for the selected media (e.g. 
multimedia, web, print). 


C) use appropriate applications including, but not 

limited to, spreadsheets and databases to develop 
charts and graphs by using data from various 
sources. 


56. You have decided to have your 
students determine which of three 
cities has the greatest rainfall during 
the month of February, and provide 
the results in the form of a graph. 
Which of the following would be the 
best type of software for the students 
to use? 


1 1 . Communication. The student delivers the product 
electronically in a variety of media, with appropriate 
supervision. The student is expected to: 




A) publish information in a variety of media including, 
but not limited to, printed copy, monitor display, 
Internet documents, and video; and 


33. publish information in a variety of 
media (e.g. printed, monitor display, 
web-based, video). 
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5) Technology research tools 7 . Use telecommunications and online 

. Students use technology to resources ( e . mai , ; on|ine discussions , 
locate, evaluate, and collec Web environments) t0 part i C ip ate in 

information from a variety of coNaborative pro blem-solving activities for 
sources 

_ . . , the purpose of developing solutions or 

• Students use technology products for audiences inside and outside 
tools to process data and the C | assr00 m. (4, 5) 

report results. 

• Students evaluate and select 3 . Use technology resources (e.g., 

new information resources calculators, data collection probes, videos, 
and technological educational software) for problem solving, 

innovations based on the self-directed learning, and extended 
appropriateness for specific learning activities. (5, 6) 
tasks. 

9. Determine which technology is useful 
and select the appropriate tool(s) and 
technology resources to address a variety 
of tasks and problems. (5, 6) 



B) use presentation software to communicate 
with specific audiences. 


37. use presentation software to share 
information with specific audiences. 


12. Communication. The student uses technology 
applications to facilitate evaluation of 
communication, both process and product. The 
student is expected to: 




A) select representative products to be collected 
and stored in an electronic evaluation tool; 


45. create an electronic teaching portfolio 
to evaluate my work, (see also 1 2-B 
and 1 2-C) 


B) evaluate the product for relevance to the 
assignment or task; and 


45. create an electronic teaching portfolio 
to evaluate my work, (see also 1 2-A 
and 1 2-C) 

46. evaluate electronic portfolio products. 


C) create technology assessment tools to 
monitor progress of project such as 
checklists, timelines, or rubrics. 


45. create an electronic teaching 
portfolio to evaluate my work, (see 
also 12-A and 1 2-B) 

47. create technology tools to assess 
student work (e.g., checklists, 
timelines, rubrics). 


4. Information acquisition. The student uses a variety 
of strategies to acquire information from electronic 
resources, with appropriate supervision. The 
student is expected to: 




A) apply appropriate electronic search strategies 
in the acquisition of information including 
keyword and Boolean search strategies; and 


23. use Boolean strategies for Internet 
searches. 


B) select appropriate strategies to navigate and 
access information on local area networks 
(LANs) and wide area networks (WANs), 
including the Internet and intranet, for 
research and resource sharing. 


30. access information on local area 
networks (LANs) and wide area 
networks (WANs). 

54. Which of the following is a well- known 
Internet search browser? 


5. Information acquisition. The student acquires 
electronic information in a variety of formats, with 
appropriate supervision. The student is expected 
to: 




A) acquire information including text, audio, 
video, and graphics; and 


24. use appropriate software and the 
Internet to find audio, video and 
graphics for lesson plans. 


B) use on-line help and documentation. 


25. use the Internet to find help when 1 
have a computer problem. 
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6. Information acquisition. The student evaluates the 
acquired electronic information. The student is 
expected to: 




A) apply critical analysis to resolve information 
conflicts and validate information; 


26. determine if information 1 find on the 
Internet is accurate and valid. 


B) determine the success of strategies used to 
acquire electronic information; and 


27. evaluate Internet search strategies to 
determine those that are most efficient. 


C) determine the usefulness and appropriateness of 
digital information. 


28. determine the usefulness and 

appropriateness of digital information. 


6) Technology problem-solving and 
decision-making tools 

• Students use technology 
resources for solving 
problems and making 
informed decisions. 

• Students employ technology 
in the development of 
strategies for solving 
problems in the real world. 


8. Use technology resources (e.g., 
calculators, data collection probes, 
videos, educational software) for 
problem solving, self-directed learning, 
and extended learning activities. (5, 6) 

9. Determine which technology is 
useful and select the appropriate 
tool(s) and technology resources to 
address a variety of tasks and 
problems. (5, 6) 

10. Evaluate the accuracy, relevance, 
appropriateness, comprehensiveness, 
and bias of electronic information 
sources. (6) 


8. Solving problems. The student uses research skills and 
electronic communication, with appropriate supervision, 
to create new knowledge. The student is expected to: 




A) use communication tools to participate in group 
projects; and 


42. use communication tools to participate 
in group projects. 


B) use electronic tools and research skills to build a 
knowledge base regarding a topic, task, or 
assignment. 


43. manipulate information in interactive 
digital environments (e.g. simulations, 
virtual labs, field trips). 


C) Participate with electronic communities as a 
learner, initiator, contributor, or mentor. 


44. participate in a listserv, chat, and 
bulletin board session. 


9. Solving problems. The student uses technology 
applications to facilitate evaluation of work, both 
process and product. The student is expected to: 




A) use software features, such as on-line help, to 
evaluate work progress; and 


3. use help menus for software programs. 


B) use software features, such as slide show previews, 
to evaluate final product. 


12. use software preview features to 
check work. 
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